Resolving `EXTRA_COMPONENT_DIRS` Path Issues in ESP-IDF CI/CD with GitHub Actions

Developing with ESP-IDF and integrating CI/CD workflows can sometimes lead to unexpected challenges, particularly when paths in the build environment differ between local and CI setups.

This post outlines my journey in troubleshooting and solving a path issue with EXTRA_COMPONENT_DIRS in a GitHub Actions environment, saving you time and effort.

The Problem

While working on an ESP-IDF component, I encountered a peculiar issue. Locally, everything worked flawlessly: I could navigate to ./test_apps/default_example and run idf.py build without any errors. However, the GitHub Action I set up for building the test_app in a Docker container failed.

The error was specific:

Directory specified in EXTRA_COMPONENT_DIRS doesn't exist:
  /__w/max6675/MAX6675

This error suggested a path-related issue in my GitHub Action configuration, pointing to a non-existent directory.

Here’s how my component’s file structure looked:

|   CMakeLists.txt
|   idf_component.yml
|   max6675.c
|   README.md
|
+---include
|       max6675.h
|
\---test_apps
    \---default_example
        |   CMakeLists.txt
        |   pytest_max6675.py
        |   sdkconfig.defaults
        |
        \---main
                CMakeLists.txt
                max6675_test.c

Troubleshooting Steps

To unravel this, I followed a systematic approach:

1. Checking EXTRA_COMPONENT_DIRS Setting: My initial suspicion was on the EXTRA_COMPONENT_DIRS in the CMake configuration. This setting points to additional directories for the build, and it needed to be correctly relative to the project’s root. However, the GitHub Actions environment has a different working directory structure.

2. Inspecting GitHub Actions Workflow: I then reviewed the .github/workflows/build_and_test.yml file. It was crucial to ensure that each step, particularly the checkout and build steps, was accurately configured to match the expected structure.

3. Verifying Repository Structure: Ensuring that my GitHub repository’s structure matched my local setup was next. The error message hinted at an incorrect directory reference.

4. Reviewing CMakeLists.txt Files: A deep dive into the CMakeLists.txt files, especially in test_apps/default_example, was necessary to confirm the correct referencing of the component directory and paths.

The Solution

The breakthrough came when I found a similar ESP-IDF CI/CD testing framework in this repository. It’s a CI/CD template for ESP-IDF components with various helpful features.

A crucial difference I noticed in their CMakeLists.txt was the following line:

set(EXTRA_COMPONENT_DIRS "PROJECT_SOURCE_DIR/../../../src/test_cmp")

Inspired by this, I modified my test_apps\default_example\CMakeLists.txt:

# file path: test_apps\default_example\CMakeLists.txt

cmake_minimum_required(VERSION 3.5)

- set(EXTRA_COMPONENT_DIRS ../../max6675)
+ set(EXTRA_COMPONENT_DIRS "PROJECT_SOURCE_DIR/../../..")

include($ENV{IDF_PATH}/tools/cmake/project.cmake)
add_compile_options("-Wno-format")
get_filename_component(ProjectId ${CMAKE_CURRENT_LIST_DIR} NAME)
string(REPLACE " " "_" ProjectId ${ProjectId})
project(${ProjectId})

And it works!

Conclusion

The modification to the EXTRA_COMPONENT_DIRS in the CMakeLists.txt was a critical step in resolving the GitHub Actions build issue. Importantly, this change did not disrupt the existing local build process that was already functioning without errors.

By using the PROJECT_SOURCE_DIR variable, the path dynamically adapts to the environment in which the build is occurring, whether it’s on a local machine or in a GitHub Actions runner.

This flexibility ensures that the test_apps can be successfully built both in the local development environment and in the GitHub Actions CI/CD pipeline, maintaining consistency across different build environments.

This approach highlights the importance of adaptable configurations in CI/CD workflows, especially when dealing with relative paths in project structures.