targets-pipelines

Use when writing complex targets pipelines with branching, batching, or custom target factories. Symptoms: multiple similar targets, parameter sweeps, method comparisons, simulation studies. Covers: static branching (tar_map/tar_combine), dynamic branching (pattern argument), hybrid patterns, custom target factories, pipeline debugging. Does NOT cover: basic targets usage, HPC deployment, package development workflows (see targetopia-packages.md).

Safety Notice

This listing is imported from skills.sh public index metadata. Review upstream SKILL.md and repository scripts before running.

Copy this and send it to your AI assistant to learn

Install skill "targets-pipelines" with this command: npx skills add jsperger/llm-r-skills/jsperger-llm-r-skills-targets-pipelines

Complex targets Pipelines

Branching creates multiple targets from a single definition. Choose your approach based on when iterations are known and what naming you need.

Quick Reference

TaskApproachKey Function/Pattern
Variants from known parametersStatictarchetypes::tar_map()
Aggregate static variantsStatictarchetypes::tar_combine()
Runtime-determined iterationsDynamicpattern = map(x)
All combinations (cartesian)Dynamicpattern = cross(x, y)
Batched simulation repsDynamictarchetypes::tar_rep()
Static variants + batched repsHybridtarchetypes::tar_map_rep()
Branch over row groupsDynamictarchetypes::tar_group_by()
Reusable multi-target patternFactoryCustom function + tar_target_raw()

Decision Tree

Need multiple similar targets?
│
├─ NO ──► Single target, or consider a target factory
│         for reusable patterns
│
└─ YES ─► Iterations known before tar_make()?
          │
          ├─ NO ──► DYNAMIC BRANCHING
          │         pattern = map() / cross()
          │         → topic-dynamic-branching.md
          │
          └─ YES ─► Need friendly target names in DAG?
                    │
                    ├─ YES ──► STATIC BRANCHING
                    │          tarchetypes::tar_map()
                    │          → topic-static-branching.md
                    │
                    └─ NO ───► Either works; prefer dynamic
                               for simplicity at scale

Additional decisions:

Static variants + many replications?
  └─► HYBRID: tarchetypes::tar_map_rep()
      → topic-hybrid-branching.md

Encapsulating a reusable multi-target pattern?
  └─► TARGET FACTORY: tar_target_raw()
      → topic-target-factories.md

Approach Comparison

AspectStaticDynamicHybrid
When definedBefore tar_make()During tar_make()Both
Target namesFriendly (analysis_bayesian)Hashed (analysis_3a7f2b)Mixed
Scale<1000 targets100k+ targetsModerate
ValidationFull tar_manifest()Must run to see branchesPartial
Use caseMethod comparisonSimulations, file processingMethods x replications

Static Branching Overview

Use tarchetypes::tar_map() when iterations are known at define-time and you want readable target names.

values <- tibble::tibble(
  method = rlang::syms(c("analyze_bayesian", "analyze_freq")),
  method_name = c("bayesian", "freq")
)

tarchetypes::tar_map(
  values = values,
  names = "method_name",
  targets::tar_target(result, method(data)),
  targets::tar_target(summary, summarize_result(result))
)
# Creates: result_bayesian, result_freq, summary_bayesian, summary_freq

Aggregate with tarchetypes::tar_combine():

mapped <- tarchetypes::tar_map(
  unlist = FALSE,
  values = values,
  targets::tar_target(result, method(data))
)

tarchetypes::tar_combine(
  all_results,
  mapped[["result"]],
  command = dplyr::bind_rows(!!!.x, .id = "method")
)

Key points:

  • Function names must be symbols: use rlang::syms(), not strings
  • Use unlist = FALSE when you need tar_combine() afterward
  • !!!.x splices the list of upstream results into the command

See topic-static-branching.md for details on tar_eval(), tar_sub(), and complex value construction.

Dynamic Branching Overview

Use the pattern argument when branch count depends on data or you need massive scale.

list(
  targets::tar_target(datasets, c("trial_A", "trial_B", "trial_C")),
  targets::tar_target(
    analysis,
    analyze_data(datasets),
    pattern = map(datasets)
  )
)

Pattern types:

  • map(x, y) - parallel iteration (zip)
  • cross(x, y) - cartesian product
  • slice(x, index = c(1, 3)) - specific elements
  • head(x, n = 5) / tail() / sample() - subsets

Critical: Patterns reference target names, not R objects or expressions.

# WRONG: Can't reference external objects
my_vec <- 1:5
targets::tar_target(x, process(my_vec), pattern = map(my_vec))

# CORRECT: Reference upstream target by name
list(
  targets::tar_target(params, 1:5),
  targets::tar_target(result, process(params), pattern = map(params))
)

See topic-dynamic-branching.md for iteration modes, row grouping, and batching.

Hybrid Patterns Overview

Combine static branching (for methods/variants) with dynamic branching (for replications):

values <- tibble::tibble(
  method = rlang::syms(c("method_a", "method_b")),
  method_name = c("a", "b")
)

tarchetypes::tar_map_rep(
  name = sim_result,
  command = method(simulate_data()),
  values = values,
  names = "method_name",
  batches = 10,

  reps = 100
)

See topic-hybrid-branching.md for complete patterns.

Target Factories Overview

A target factory is a function that returns pre-configured target objects:

#' @export
analyze_dataset <- function(name, file_path) {
  name <- targets::tar_deparse_language(substitute(name))
  name_data <- paste0(name, "_data")
  sym_data <- as.symbol(name_data)

  command <- substitute(analyze(data), env = list(data = sym_data))

  list(
    targets::tar_target_raw(name_data, substitute(load_csv(file_path))),
    targets::tar_target_raw(name, command)
  )
}

See topic-target-factories.md for metaprogramming patterns and validation.

Debugging and Troubleshooting

Validation Commands

# Check target definitions before running
targets::tar_manifest()
targets::tar_manifest(fields = c("name", "command", "pattern"))

# Visualize DAG (slow with many targets)
targets::tar_visnetwork(targets_only = TRUE)

# Test pattern logic without running pipeline
targets::tar_pattern(
  cross(method, map(dataset, seed)),
  method = 3,
  dataset = 5,
  seed = 10
)

Common Errors

ErrorCauseFix
"Target 'x' not found" in patternPattern references non-existent targetCheck spelling; ensure upstream target exists
"object 'x' not found" in commandExternal object in patternCreate upstream target; patterns reference target names only
Strings not callableUsing c("func_a", "func_b")Use rlang::syms() for function names
Can't selectively combineMissing unlist = FALSEAdd to tar_map() when using tar_combine()
Cryptic target namesDynamic branching defaultAdd metadata to output or use static branching

Identifying Failed Branches

# Check which targets have errors
targets::tar_meta() |>
  dplyr::filter(error != "")

# Read specific branch by index
targets::tar_read(target_name, branches = 1)

# Inspect all branch metadata
targets::tar_branches(target_name)

Debugging Strategies

  1. Start with tar_manifest() - verify target count and commands before running
  2. Test with subset - use pattern = head(x, n = 3) for initial runs
  3. Include provenance - add parameter values to branch outputs for traceability
  4. Check upstream first - use tar_read() on dependencies before investigating branching

Common Mistakes

MistakeProblemFix
pattern = map(df$col)Can't reference external objectsCreate upstream target
method = c("fn_a", "fn_b")Strings aren't callablerlang::syms(c("fn_a", "fn_b"))
Manual lapply + aggregationReinventing batchingUse tar_rep() or tar_map_rep()
tar_make() without validationErrors discovered lateAlways tar_manifest() first
expand_grid() with syms() insidesyms() doesn't expand in gridApply syms() after grid creation

See Also

  • r-metaprogramming: Expression manipulation for target factories
  • tidy-evaluation: Programming with data-masked functions
  • targetopia-packages.md: Package development workflows

Reference Files

External Resources

Source Transparency

This detail page is rendered from real SKILL.md content. Trust labels are metadata-based hints, not a safety guarantee.

Related Skills

Related by shared tags or category signals.

Coding

metaprogramming

No summary provided by upstream source.

Repository SourceNeeds Review
General

ggplot2

No summary provided by upstream source.

Repository SourceNeeds Review
General

hardhat

No summary provided by upstream source.

Repository SourceNeeds Review
General

tidymodels-overview

No summary provided by upstream source.

Repository SourceNeeds Review