jq-transforms

Overview

jq-transforms is a schema-driven JSON transformation library built specifically for AI agents. It provides 60+ composable transformations for JSON manipulation, each with clear input/output contracts, comprehensive tests, and minimal implementations. The library emphasizes discoverability through schemas and composability through Unix pipes.

When to Use This Skill

Use this skill when:

• Transforming JSON data structures (CRUD operations, nested paths)

• Filtering or mapping arrays within JSON objects

• Extracting specific fields or patterns from JSON

• Validating JSON schemas or field types

• Converting between JSON types

• Composing multi-step JSON transformations

• Processing API responses or configuration files

Core Transformation Categories

CRUD Operations (crud-*)

Path-based object manipulation for getting, setting, deleting, checking existence, merging, and querying nested JSON structures.

Common examples:

• crud-get : Retrieve value at nested path with optional default

• crud-set : Set value at path, creating intermediate objects

• crud-delete : Remove field at path

• crud-has : Check if path exists

• crud-merge : Recursively merge two objects

Array Operations (array-*)

Array transformations including filtering, mapping, reducing, flattening, chunking, and deduplication.

Common examples:

• array-filter : Filter array by field value

• array-map : Extract field from each array element

• array-reduce : Aggregate array (sum, avg, min, max, count)

• array-unique : Remove duplicate values

• array-flatten : Flatten nested arrays to specified depth

String Operations (string-*)

String manipulation within JSON objects.

Common examples:

• string-split : Split string into array by separator

• string-join : Join array into string with separator

• string-replace : Replace pattern in string

• string-trim : Remove whitespace from string

Extraction (extract-*)

Pattern-based extraction from text fields.

Common examples:

• extract-urls : Extract URLs from text

• extract-code-blocks : Extract code blocks from markdown

• extract-mentions : Extract @mentions from text

Validation (validate-, is-, has-*)

Field validation and type checking.

Common examples:

• validate-required : Check required fields exist

• validate-types : Validate field types

• is-timestamp : Check if value is valid timestamp

• has-field : Check if nested path exists

Type Conversions (to-*)

Convert between JSON types safely.

Common examples:

• to-string , to-number , to-boolean , to-array , to-object

Usage Pattern

All transformations follow a consistent invocation pattern:

jq -f <transform-dir>/transform.jq [--arg key val] [--argjson key json] input.json

Example - Get nested value:

echo '{"user":{"name":"skogix"}}' |
jq -f crud-get/transform.jq --arg path "user.name"

Output: "skogix"

Example - Filter array:

echo '{"items":[{"status":"active"},{"status":"inactive"}]}' |
jq -f array-filter/transform.jq
--arg array_field "items"
--arg field "status"
--arg value "active"

Output: {"items":[{"status":"active"}]}

Example - Compose multiple transforms:

cat data.json |
jq -f array-filter/transform.jq --arg array_field "users" --arg field "active" --arg value "true" |
jq -f array-map/transform.jq --arg array_field "users" --arg field "email"

Transform Discovery

To find the right transform for a task:

• By category: Refer to the categories above to narrow down the domain (CRUD, array, string, etc.)

• By cheat sheet: Consult references/CHEAT_SHEET.md for complete transform index with arguments and examples

• By schema: Read <transform-dir>/schema.json for detailed input/output contracts

Each transform directory contains:

• transform.jq

• The jq implementation (5-40 lines)

• schema.json

• Input/output contract with examples

• test.sh

• Comprehensive test suite

• test-input-*.json

• Test fixtures

Key Design Principles

• Schema-first: Every transform has explicit input/output/args contract

• Argument-based: All parameters via --arg /--argjson (no hardcoding)

• Composable: Chain transforms via Unix pipes

• Type-safe: Graceful handling of missing paths and wrong types

• Self-contained: Each transform is isolated, no dependencies

• Minimal: Fewer lines = fewer bugs, easier to understand

Common Patterns

Get value with default:

jq -f crud-get/transform.jq --arg path "config.timeout" --arg default "30"

Set nested value:

jq -f crud-set/transform.jq --arg path "user.profile.age" --arg value "30"

Filter and extract:

jq -f array-filter/transform.jq --arg array_field "users" --arg field "active" --arg value "true" |
jq -f array-map/transform.jq --arg array_field "users" --arg field "email"

Validate required fields:

jq -f validate-required/transform.jq --arg fields "name,email,age"

Critical Implementation Notes

When creating new transforms or debugging issues:

Avoid common jq pitfalls:

• Use try-catch instead of // fallback for falsy value handling

• Use has() instead of != null for existence checks

• Check array types before using map() operations

Refer to references/IMPLEMENTATION_SPEC.md for complete implementation patterns, test coverage requirements, and detailed pitfall documentation.

References

This skill bundles comprehensive reference documentation (loaded only when needed):

• CHEAT_SHEET.md - Quick reference of all 60+ transforms with arguments and examples

• IMPLEMENTATION_SPEC.md - Complete implementation guide with patterns and pitfalls

• README.md - User-facing documentation and design principles

• USAGE_EXAMPLES.md - Real-world usage scenarios

Testing

Test transforms before use:

Test specific transform

./crud-get/test.sh

Test all transforms

./test-all.sh

Each transform has 8-17 test cases covering happy paths, falsy values, type safety, boundary conditions, and error cases.