Skip to content

Environment

The env object passed to the validate(env), build(env) and post_build(env) steps gives access to options, queries and collectors, and performs all file operations. The validate step can only read the environment, the build step can generate files and add to collectors, and the post-build step can additionally read the build log, but not add to collectors anymore.

See Modules for the validate step and Patching Generated Files for the patch argument.

Build

# The build step can do everything the validation step can
# But now you can finally generate files
def build(env):
    # Set the output base path, this is relative to the output path, which
    # defaults to the current working directory (see `lbuild -p` and `<outpath>`)
    env.outbasepath = "repo/module"

    # Copy single files, the source path is relative to the module file.
    # You can only copy files from inside your repository.
    env.copy("file.hpp")
    # Copy single files while renaming them
    env.copy("file.hpp", "cool_filename.hpp")
    # Relative paths are preserved!!!
    env.copy("../file.hpp") # copies to repo/file.hpp
    env.copy("../file.hpp", dest="file.hpp") # copies to repo/module/file.hpp

    # You can also copy entire folders
    env.copy("folder/", dest="renamed/")
    # and ignore specific RELATIVE files/folders
    env.copy("folder/", ignore=env.ignore_files("*.txt", "this_specific_file.hpp"))
    # or ignore specific ABSOLUTE paths
    env.copy("folder/", ignore=env.ignore_paths("*/folder/*.txt"))
    # Apply a unified diff to the copied files. The paths inside the patch are
    # relative to `env.outbasepath`, so here `renamed/file.hpp` etc.
    # The build fails if the patch does not apply cleanly or if it contains
    # files that were not generated by this operation.
    env.copy("folder/", dest="renamed/", patch="fix_folder.patch")

    # You can also copy files out of a .zip or .tar archive
    env.extract("archive.zip") # everything inside the archive
    env.extract("archive.zip", dest="renamed/") # extract into folder
    # You can extract only parts of the archive, like a single file
    env.extract("archive.zip", src="filename.hpp", dest="renamed.hpp")
    # or a single folder somewhere in the archive. Note that folders inside the
    # archive must end with a `/`!
    env.extract("archive.zip", src="folder/subfolder/", dest="renamed/folder")
    # of course, you can ignore files and folders inside the archive too
    env.extract("archive.zip", src="folder/", dest="renamed", ignore=env.ignore_files("*.txt"))

    # All file operations return a set of build log operations, which you can
    # use to attach collector values to specific files, see Collectors.
    operations = env.copy("folder/")

    # Set the global Jinja2 substitutions dictionary
    env.substitutions = {
        "hello": "world",
        "instances": list(map(str, env["repo:instances"])),
        "build_data": build_data, # from validation step
    }
    # and generate a file from a template
    env.template("template.hpp.in")
    # any `.in` postfix is automatically removed, unless you rename it
    for instance in env["repo:instances"]:
        env.template("template.hpp.in", "template_{}.hpp".format(instance))
    # You can explicitly add Jinja2 substitutions and filters.
    # Note: `env.substitutions` overwrite substitutions of the same name!
    env.template("template.hpp.in",
                 substitutions={"more": "subs"},
                 filters={"stringify": lambda i: str(i)})
    # Note: these filters are NOT namespaced with the repository name!
    # The rendered file can be patched too
    env.template("template.hpp.in", patch="fix_template.patch")

    # submodules are build first, so you can access the generated files
    # These paths are relative to `env.outbasepath`.
    headers = env.generated_local_files(lambda file: file.endswith(".hpp"))
    # and use this information for a new template.
    env.template("module_header.hpp.in", substitutions={"headers": headers})

    # Add values to a collector, all these are type checked
    env.collect("::include_path", "repo/must_be_valid_path/", "repo/folder2/")

Paths

The environment provides helpers to relocate paths, which is useful for passing paths to generated build system files:

def build(env):
    env.outbasepath = "repo/module"
    # Absolute path inside the output path including `env.outbasepath`
    env.real_outpath("file.hpp")                # /abs/outpath/repo/module/file.hpp
    env.real_outpath("file.hpp", basepath=".")  # /abs/outpath/file.hpp
    # Path relative to the output path or to a folder inside it
    env.relative_outpath("repo/module/file.hpp", "repo")   # module/file.hpp
    # Absolute path inside the output path ignoring `env.outbasepath`
    env.cwdoutpath("build")                     # /abs/outpath/build
    # Path inside the output path relative to the current working directory
    env.relcwdoutpath("build")                  # outpath/build
    # Absolute path inside the current working directory
    env.cwdpath("project.xml")                  # /abs/cwd/project.xml

To reference files in the repository or module folder use the global localpath() and repopath() functions, see Files.

Patching Generated Files

Sometimes a module needs to fix files it copies from a third-party source, which is better kept unmodified. Pass a unified diff to env.copy(...) or env.template(...) with the patch argument and lbuild applies it to the files that this operation just generated:

def build(env):
    env.outbasepath = "repo/ext/printf"
    # Copy the unmodified upstream file and patch the copy
    env.copy("printf/src/printf.c", "printf.c", patch="printf.c.patch")
    # Patches can also span several files of a folder copy
    env.copy("printf/src/", dest="src/", patch="printf_src.patch")

The patch path is relative to the module file, just like src. The paths inside the patch are relative to env.outbasepath, just like dest, so for the first operation above the patch must look like this:

diff --git a/printf.c b/printf.c
--- a/printf.c
+++ b/printf.c
@@ -329,3 +329,8 @@
 // behavior with -LONG_MIN or -LLONG_MIN
 #define ABS_FOR_PRINTING(_x) ((printf_unsigned_value_t) ( (_x) > 0 ? (_x) : -((printf_signed_value_t)_x) ))

+__attribute__((weak)) void putchar_(char c)
+{
+  (void) c;
+}
+

The easiest way to create such a patch is to modify the source file in place and let git generate the diff relative to the source folder:

git diff --relative=printf/src -- printf/src/printf.c > printf.c.patch
git checkout -- printf/src/printf.c

The patch is applied by libgit2 (via pygit2), so no git installation is required. Patching is strict and fails the build with an error, if:

  • a hunk does not match the file at exactly its line numbers. There is no fuzzy matching or offset search, so you must update the patch whenever the source changes around a hunk.
  • the patch is already applied, which usually means it was upstreamed.
  • the patch modifies a file that was not generated by this operation, except for files of a folder copy that were excluded by the ignore filter.
  • the patch creates, deletes or renames files.
  • the patch is empty or malformed.

This lets you keep one patch for an entire folder, even if a module only copies a few files out of it. Diffs of ignored files are still applied to their source files in memory, so a patch that no longer applies fails the build, even if the patched file is not copied:

def build(env):
    # Only copy the header for this device, but keep one patch for all devices
    headers = ["stm32g474xx.h", "stm32g4xx.h", "system_stm32g4xx.h"]
    env.copy("stm32g4xx/Include", dest=".", patch="patches/stm32g4xx.patch",
             ignore=lambda path, files: [f for f in files if f not in headers])

Symlinked copies (lbuild build --symlink) are replaced by regular files before patching, so the source files are never modified.

Post-Build

# The post build step can do everything the build step can,
# but you can't add to the metadata anymore:
# - env.collect() unavailable
# You have access to the entire buildlog up to this point
def post_build(env):
    # The absolute path to the lbuild output directory
    outpath = env.buildlog.outpath

    # All modules that were built
    modules = env.buildlog.modules
    # All file generation operations that were done
    operations = env.buildlog.operations
    # All operations per module
    operations = env.buildlog.operations_per_module("repo:module")

    # iterate over all operations directly
    for operation in env.buildlog:
        # Get the module name that generated this file
        env.log.info("Module({}) generated the '{}' file"
                     .format(operation.module, operation.filename))
        # You can also get the filename relative to a subfolder in outpath
        env.relative_outpath(operation.filename, "subfolder/")
        # or as an absolute path
        env.real_outpath(operation.filename, "subfolder/")

    # get all include paths from all active modules
    include_paths = env.collector_values("::include_path")