Operations
Operations transform files in the pipeline's staging area. They run sequentially, in the order they appear in the pipeline script, after the SOURCE and TARGET blocks.
Key concepts
Source vs staging. COPY is the only operation that reads from workspace/source/ (the
clone of the SOURCE endpoint). All other operations read from and write to workspace/staging/
— the accumulating output that will eventually be overlaid onto the TARGET.
Glob patterns. Patterns follow Python's pathlib.Path.glob convention: * matches any
sequence of characters within a single path segment; ** matches zero or more path segments
recursively. Patterns are always matched relative to the staging directory root.
Error behaviour. Operations that must match at least one file — COPY, RENAME, DELETE,
MOVE, CHMOD — raise an error and abort the run if nothing matches. Operations that apply
a transformation to whatever they find — REPLACE, REGEX_REPLACE, STRIP_LINES, APPEND,
PREPEND, TOML_UNSET, YAML_UNSET — silently succeed even when no files or keys match.
Format-aware operations (TOML_UNSET, YAML_UNSET) can be made to fail on missing keys
by providing the strict argument.
Text encoding. All text-based operations (REPLACE, REGEX_REPLACE, STRIP_LINES,
APPEND, PREPEND) read and write files as UTF-8. Bytes that cannot be decoded are
substituted with the replacement character (U+FFFD) rather than causing an error.
Format-aware operations. TOML_SET, TOML_UNSET, TOML_APPEND_TABLE, YAML_SET,
YAML_UNSET, and YAML_APPEND parse structured files and modify only the targeted keys,
leaving comments and unrelated content intact. key_path is a dot-separated sequence of
keys (section.subsection.key); missing intermediate tables or mappings are created
automatically. In TOML_SET and YAML_SET, the value argument is first attempted to be
parsed as JSON; if successful, the resulting structure (object, array, etc.) is
assigned to the key. If JSON parsing fails, the value is coerced to the most specific
scalar type in order: true/false → bool, integer literals → int, decimal literals →
float, anything else → string. TOML_APPEND_TABLE and YAML_APPEND also accept a
JSON-encoded value as their third argument, which is parsed at execution time —
invalid JSON raises an error before the file is touched.
Quoting arguments. Arguments that contain spaces must be wrapped in double quotes. Inside
double-quoted arguments only " and \ can be escaped with a backslash. \n is not
interpreted as a newline — it is the two-character literal string \n.
COPY
Copies entries matching src_glob from workspace/source/ into workspace/staging/dest_path/,
preserving the relative path of every matched entry.
Files: each matched file is placed at staging/<dest_path>/<relative-path-in-source>.
Parent directories are created as needed.
Directories: when a matched entry is a directory, its entire subtree is merged into
staging/<dest_path>/<relative-path-in-source>/ using a recursive copy. Files in the
destination that already exist are overwritten; others are left in place.
Errors: fails if src_glob matches nothing.
# Copy every Python file from src/, placing them under lib/ in staging.
# A file at src/utils/helpers.py lands at staging/lib/utils/helpers.py.
COPY src/**/*.py lib/
# Copy an entire directory tree; assets/icons/ appears at staging/dist/icons/.
COPY assets/icons dist/
# Copy a single file to the staging root.
COPY pyproject.toml .
RENAME
Renames files in staging whose filename or relative path matches the glob pattern. The first
occurrence of pattern in the matched portion is replaced with replacement. Directories
themselves are not renamed — only files are affected.
When the full relative path (e.g., tests/test_utils.py) matches pattern, the rename is
applied to the full path. When only the filename (e.g., test_utils.py) matches, the rename
is applied to the filename only. In both cases the file is moved to its new path and parent
directories are created if needed.
Errors: fails if no files in staging match pattern.
# Rename any file or path segment containing "test_" to "spec_".
# tests/test_utils.py → tests/spec_utils.py
RENAME test_ spec_
# Change the file extension of all shell scripts.
# deploy.sh → deploy.bash
RENAME .sh .bash
REPLACE
Performs a literal (non-regex) text substitution in all staging files matching
file_glob. Every occurrence of the exact string find is replaced with replace.
Files that do not contain find are left unchanged and do not trigger an error.
Default file_glob: **/* (all files).
Errors: never fails due to no matches. Fails if a file cannot be read or written.
# Pin the version string in all TOML files.
REPLACE "version = \"dev\"" "version = \"1.0.0\"" **/*.toml
# Replace a package import path across all Python files.
REPLACE "from internal.utils" "from mylib.utils" **/*.py
# Replace across all files (no glob needed for the default).
REPLACE "http://staging.internal" "https://api.example.com"
DELETE
Removes all files in staging that match glob. Only files are removed; empty directories
left behind are not cleaned up automatically.
Errors: fails if glob matches no files.
# Remove compiled Python bytecode.
DELETE **/__pycache__/**
# Remove a specific generated file.
DELETE src/_version.py
# Remove all Markdown files from a subdirectory.
DELETE docs/**/*.md
MOVE
Moves files and directories in staging that match src_glob to dest_path, replicating the
behaviour of the Unix mv command. Directories are moved recursively. The original entries are
removed.
Destination semantics:
- If the glob matches a single entry and dest_path does not yet exist, the entry is
renamed/moved to exactly dest_path (rename semantics).
- Otherwise dest_path is created as a directory and each matched entry is placed inside it
by its own name.
When a recursive glob (e.g. build/**) matches both a directory and its children, only the
directory is moved; its children are not moved separately.
Errors: fails if src_glob matches no entries.
# Rename a directory.
MOVE old_name new_name
# Move a directory into an existing directory (result: output/subdir/).
MOVE subdir output/
# Move top-level scripts into a bin/ directory.
MOVE *.sh bin/
APPEND
Appends text to the end of every staging file matching file_glob. The text is written
exactly as provided — no newline is added automatically, and \n in the argument is the
literal two-character sequence backslash-n, not a newline character.
Default file_glob: **/* (all files).
Errors: never fails due to no matches. Fails if a file cannot be written.
# Append a trailing comment to every Python file.
APPEND "# end of file" **/*.py
# Append an attribution line to a specific file (no trailing newline added).
APPEND "Generated by remanufacture." README.md
PREPEND
Prepends text to the beginning of every staging file matching file_glob. The text is
written exactly as provided. No separator is added automatically.
Default file_glob: **/* (all files).
Errors: never fails due to no matches. Fails if a file cannot be read or written.
# Add a licence header to every shell script.
PREPEND "# Copyright 2025 My Org. All rights reserved." **/*.sh
# Prepend a shebang to entry-point scripts.
PREPEND "#!/usr/bin/env python3" bin/*
REGEX_REPLACE
Replaces all occurrences of the Python regular expression pattern with replacement in
every staging file matching file_glob. replacement may reference captured groups using
\1, \2, etc. Files where the pattern does not match are left unchanged.
Default file_glob: **/* (all files).
Errors: fails immediately if pattern is not a valid Python regex. Never fails due to
no file matches.
# Remove all TODO comment lines from Python files.
REGEX_REPLACE "^#\s*TODO.*\n" "" **/*.py
# Replace semantic version references (e.g., 1.2.3 → 2.0.0).
REGEX_REPLACE "\d+\.\d+\.\d+" "2.0.0" **/*.md
# Strip trailing whitespace from every line in text files.
REGEX_REPLACE "[ \t]+$" "" **/*.txt
STRIP_LINES
Removes every line in matching staging files for which the Python regular expression pattern
finds a match anywhere in that line. Line endings are preserved for the surviving lines.
Default file_glob: **/* (all files).
Errors: fails immediately if pattern is not a valid Python regex. Never fails due to
no file matches.
# Remove lines that contain a debug logging call.
STRIP_LINES "logger\.debug" **/*.py
# Strip lines that import a deprecated module.
STRIP_LINES "^import old_compat" **/*.py
# Remove blank lines from configuration files.
STRIP_LINES "^\s*$" **/*.cfg
CHMOD
Sets the file permissions of every staging file matching glob to mode. The mode must be
an octal integer literal (e.g., 0755, 0644). Only files are affected; directories are
ignored.
Errors: fails if glob matches no files or if mode is not a valid octal literal.
# Make all shell scripts executable.
CHMOD 0755 bin/*.sh
# Ensure library files are read-only for group and other.
CHMOD 0644 lib/**/*.py
# Lock down a generated secrets file.
CHMOD 0600 dist/.env
TOUCH
Creates a file at path inside staging, writing content into it. If the file already
exists it is overwritten. Parent directories are created as needed. When content is
omitted, an empty file is created.
content is a single argument and must be quoted if it contains spaces. Like all text
arguments, \n is the literal two-character sequence backslash-n, not a newline.
# Create an empty marker file.
TOUCH dist/.nojekyll
# Write a static version string into a file.
TOUCH src/_version.py "__version__ = \"1.0.0\""
# Create a MANIFEST with a short note.
TOUCH dist/MANIFEST "auto-generated"
PAUSE
Suspends execution for seconds seconds. Accepts integer or decimal values.
This operation is intended for debugging only. Insert a PAUSE to hold the run while you inspect the contents of the temporary workspace before subsequent operations execute. Remove it before committing a final pipeline script.
Errors: fails if seconds is not a valid number or is negative.
# Pause for 60 seconds — enough time to inspect workspace/staging/ manually.
PAUSE 60
# Short pause to observe operation order.
PAUSE 2.5
TOML_SET
Sets or creates a key in a TOML file in staging. file_path is relative to the
staging directory root. key_path is a dot-separated path to the target key
(section.subsection.key). Any missing intermediate tables are created automatically as
regular [tables].
The value argument is first attempted to be parsed as JSON. If successful, the
resulting structure (such as a dictionary or a list) is assigned to the key. This allows
setting complex nested structures directly.
If JSON parsing fails, value is coerced to the most specific scalar type: true/false
→ bool, integer literals → int, decimal literals → float, anything else remains a string.
When the JSON value is an array of objects, each object is written as a TOML inline
table, producing an inline array (e.g. authors = [{name = "Jane Doe"}]). To append
entries to a TOML array of tables ([[key]]) use TOML_APPEND_TABLE instead.
Errors: fails if file_path does not exist in staging or the file cannot be parsed as
valid TOML.
# Pin the project version.
TOML_SET pyproject.toml project.version "2.0.0"
# Set a complex data structure using JSON.
TOML_SET config.toml data '{"threshold": 0.5, "modes": ["fast", "safe"]}'
# Enable a boolean flag in a nested section, creating it if absent.
TOML_SET config.toml tool.formatter.enabled true
# Set a PEP 621 authors list (inline array of objects).
TOML_SET pyproject.toml project.authors '[{"name":"Jane Doe","email":"jane@example.com"}]'
TOML_UNSET
Removes a key or an entire section from a TOML file in staging. When key_path points to
a [table], the whole section including all its keys and subsections is removed.
By default, if the specified key_path is not found, the operation silently succeeds. If
the optional strict argument is provided, the operation will fail if any segment of the
key path is missing.
Errors: fails if file_path does not exist in staging, the file cannot be parsed as
valid TOML, or key_path is missing and strict is specified. Providing a third argument
other than strict is a parse-time error.
# Remove a stale plugin section. If missing, no error is raised.
TOML_UNSET pyproject.toml tool.old_plugin
# Fail the pipeline if a required configuration key cannot be removed.
TOML_UNSET config.toml server.deprecated_flag strict
TOML_APPEND_TABLE
Appends a JSON object as a new entry to the [[key_path]] array of tables in a TOML file
in staging. If the key does not exist, a new array of tables is created. json_object must
be a valid JSON object (not an array or scalar).
Errors: fails if file_path does not exist in staging, json_object is not a valid
JSON object, or key_path already holds a regular [table] rather than an array of tables.
# Add a server entry to [[servers]].
TOML_APPEND_TABLE config.toml servers '{"host": "alpha", "port": 8080}'
# Add a second entry (appended after the first).
TOML_APPEND_TABLE config.toml servers '{"host": "beta", "port": 8081}'
YAML_SET
Sets or creates a key in a YAML file in staging. file_path is relative to the
staging directory root. key_path is a dot-separated path to the target key. Any missing
intermediate mappings are created automatically. Comments and surrounding formatting are
preserved.
The value argument is first attempted to be parsed as JSON, otherwise it is coerced
using the same rules as TOML_SET.
Errors: fails if file_path does not exist in staging or the file cannot be parsed as
valid YAML.
# Update the runner image for a GitHub Actions job.
YAML_SET .github/workflows/ci.yml jobs.test.runs-on ubuntu-24.04
# Set a nested mapping using JSON.
YAML_SET values.yaml ingress.annotations '{"cert-issuer": "letsencrypt", "force-ssl": "true"}'
# Set a numeric replica count.
YAML_SET values.yaml replicaCount 3
YAML_UNSET
Removes a key or an entire subtree from a YAML file in staging. When key_path points to a
mapping, the whole mapping including all nested keys is removed. Comments and surrounding
formatting are preserved.
By default, if the specified key_path is not found, the operation silently succeeds. If
the optional strict argument is provided, the operation will fail if any segment of the
key path is missing.
Errors: fails if file_path does not exist in staging, the file cannot be parsed as
valid YAML, or key_path is missing and strict is specified. Providing a third argument
other than strict is a parse-time error.
# Remove a TLS subtree from a Helm values file.
YAML_UNSET values.yaml ingress.tls
# Fail if a specific job configuration is missing and cannot be removed.
YAML_UNSET .github/workflows/ci.yml jobs.test.continue-on-error strict
YAML_APPEND
Appends a JSON-encoded value to the YAML sequence at key_path in a YAML file in staging.
If the key does not exist, a new sequence containing only the appended value is created.
json_value may be any valid JSON type (object, array, string, number, boolean).
Errors: fails if file_path does not exist in staging, json_value is not valid JSON,
or the existing value at key_path is not a sequence.
# Add an image pull secret entry to a Helm values file.
YAML_APPEND values.yaml imagePullSecrets '{"name": "regcred"}'
# Append a string to a list of allowed hosts.
YAML_APPEND config.yaml server.allowed_hosts '"new.example.com"'
# Append a plain integer to a list.
YAML_APPEND config.yaml allowed_ports 8443