Lessons

Developing Code for Data Processing

Reference card — pipeline control-flow operators

Create a pipeline component that uses control flow operators (e.g., if/else, for/each, etc.).

Reference card. The control-flow operators are Job-level and were built in depth in [Jobs & orchestration — multi-task, dependencies, control flow](/lessons/s1-jobs-orchestration/) — this ties them off with the exam tells. One idea: a Job isn't just a straight chain; it can branch and fan out.

The operators

OperatorWhat it doesExam tell
depends_ontask B runs only after task A succeeds; independent tasks run in parallelenforce order / express a DAG
Condition task (if/else)branches on a boolean (usually from a task value); the untaken branch is skipped, not failed"route to X or Y based on a computed value" (e.g. quality_score > 0.95 → load, else quarantine)
For-each taskruns one task template over a list, instances in parallel (with a concurrency limit)"run the same logic over N inputs/regions/report-types"
outcome: "failed" dependencya downstream task that runs only when upstream failedalerts / cleanup on failure

The two tells that get tested

That's the whole surface. Depth, cluster choices, task values, and the no-cross-task-rollback rule all live in [Jobs & orchestration — multi-task, dependencies, control flow](/lessons/s1-jobs-orchestration/).

Recall (say without scrolling up)

  1. "Same logic over 15 report types, in parallel" — which operator?
  2. "Branch to load-or-quarantine on a computed score" — which operator, and what happens to the branch not taken?
  3. Where does the value the condition task reads come from, and what type does it arrive as?

Prerequisites