Fluent Bit Config Generator

Trigger Guidance

Use this skill when the user asks for any of the following:

• Create or update a Fluent Bit config (fluent-bit.conf , parsers.conf )

• Build a log pipeline (INPUT -> FILTER -> OUTPUT)

• Configure Kubernetes logging with metadata enrichment

• Send logs/metrics to Elasticsearch, Loki, CloudWatch, S3, Kafka, OTLP, or Prometheus remote write

• Implement parser, multiline, lua, or stream processing behavior

Do not use this skill for pure validation-only requests; use fluentbit-validator in that case.

Execution Flow

Follow this sequence exactly. Do not skip stages.

Stage 1: Set Working Context and Preflight

Use one of these deterministic command patterns.

Option A (recommended): run from this skill directory

cd /Users/akinozer/GolandProjects/cc-devops-skills/devops-skills-plugin/skills/fluentbit-generator python3 scripts/generate_config.py --help

Option B: run from any cwd with absolute paths

python3 /Users/akinozer/GolandProjects/cc-devops-skills/devops-skills-plugin/skills/fluentbit-generator/scripts/generate_config.py --help

Preflight checks:

• Confirm python3 is available.

• Confirm script help loads without errors.

• If using relative output paths, run from the intended target cwd.

Fallback:

• If python3 or script execution is unavailable, switch to manual generation flow (Stage 4) using examples/ templates.

Stage 2: Clarification Questionnaire (Explicit Template)

Collect these fields before generation.

Required questions:

• Primary use case: kubernetes , application logs , syslog , http ingest , metrics , or custom ?

• Inputs: which sources and paths/ports (for example tail /var/log/containers/*.log )?

• Outputs: destination plugin(s) and endpoint(s) (host, port, uri/topic/index/bucket)?

• Reliability/security requirements: TLS on/off, retry expectations, buffering limits?

• Environment context: cluster name, environment name, cloud region (if applicable)?

Optional but important:

• Expected log format: json, regex, cri, docker, multiline stack traces?

• Throughput profile: low/medium/high and acceptable latency?

• Constraints: offline environment, missing binaries, read-only filesystem?

If any required answer is missing, ask focused follow-up questions before generating.

Stage 3: Decide Script vs Manual Generation

Use this decision table.

Condition Path

Request matches a built-in use case and only needs supported flags Script path (Stage 5)

Request needs uncommon plugin options not represented by script flags Manual path (Stage 4)

Complex multi-filter chain, custom parser/lua logic, or specialized plugin tuning Manual path (Stage 4)

Script cannot run in environment (missing Python/permissions) Manual path (Stage 4)

User explicitly requests hand-crafted config Manual path (Stage 4)

Supported script use cases:

• kubernetes-elasticsearch

• kubernetes-loki

• kubernetes-cloudwatch

• kubernetes-opentelemetry

• application-multiline

• syslog-forward

• file-tail-s3

• http-kafka

• multi-destination

• prometheus-metrics

• lua-filtering

• stream-processor

• custom

Always state the decision explicitly, including why the other path was not used.

Stage 4: Manual Generation (When Script Is Not the Best Fit)

• Read the closest template in examples/ first.

• Read examples/parsers.conf before defining new parsers.

• Assemble config in this order:

• [SERVICE]

• [INPUT]

• [FILTER]

• [OUTPUT]

• parser definitions (if needed)

• Reuse known-good sections from examples and only customize required parameters.

• Keep tags and parser references consistent across sections.

Required local template selection:

• Kubernetes + Elasticsearch: examples/kubernetes-elasticsearch.conf

• Kubernetes + Loki: examples/kubernetes-loki.conf

• Kubernetes + CloudWatch: examples/cloudwatch.conf

• Kubernetes + OpenTelemetry: examples/kubernetes-opentelemetry.conf

• App multiline: examples/application-multiline.conf

• Syslog forward: examples/syslog-forward.conf

• File to S3: examples/file-tail-s3.conf

• HTTP to Kafka: examples/http-input-kafka.conf

• Multi destination: examples/multi-destination.conf

• Metrics: examples/prometheus-metrics.conf

• Lua filtering: examples/lua-filtering.conf

• Stream processor: examples/stream-processor.conf

• Parsers: examples/parsers.conf

Fallback:

• If no matching example exists, start from scripts/generate_config.py --use-case custom output shape and extend manually.

Stage 5: Script Generation Commands (Deterministic Examples)

Run from skill directory unless output path is absolute.

cd /Users/akinozer/GolandProjects/cc-devops-skills/devops-skills-plugin/skills/fluentbit-generator

Kubernetes -> Elasticsearch

python3 scripts/generate_config.py
--use-case kubernetes-elasticsearch
--cluster-name prod-cluster
--environment production
--es-host elasticsearch.logging.svc
--es-port 9200
--output output/fluent-bit.conf

Kubernetes -> OpenTelemetry

python3 scripts/generate_config.py
--use-case kubernetes-opentelemetry
--cluster-name prod-cluster
--environment production
--otlp-endpoint otel-collector.observability.svc:4318
--output output/fluent-bit-otlp.conf

File tail -> S3

python3 scripts/generate_config.py
--use-case file-tail-s3
--log-path /var/log/app/*.log
--s3-bucket my-logs-bucket
--s3-region us-east-1
--output output/fluent-bit-s3.conf

Fallback:

• If a required option is unsupported by the script, document that gap and switch to Stage 4.

Stage 6: Plugin Documentation Lookup Fallback Chain

Use this strict order when plugin behavior or parameters are uncertain.

• Context7 (first choice)

• Resolve library id for Fluent Bit docs.

• Query plugin-specific configuration details.

• Official Fluent Bit docs (second choice)

• Use https://docs.fluentbit.io/manual and plugin-specific pages.

• Prefer official plugin reference sections over blogs.

• Web search (last choice)

• Use only when Context7 and official docs are unavailable/incomplete.

• Prioritize sources that quote or link official docs.

When to stop escalating:

• Stop as soon as required parameters and one validated example are found.

• If all sources are unavailable, proceed with local examples/ and clearly mark assumptions.

Stage 7: Validation and Fallback Behavior

Primary validation path:

• Invoke fluentbit-validator after generation.

If validator is unavailable, run local fallback checks:

Syntax/format smoke check when fluent-bit is installed

fluent-bit -c <generated-config> --dry-run

Optional execution test

fluent-bit -c <generated-config>

If fluent-bit binary is missing:

• Perform static checks manually:

• section headers are valid ([SERVICE] , [INPUT] , [FILTER] , [OUTPUT] )

• required plugin keys exist

• tag/match patterns align

• parser names/files referenced correctly

• no hardcoded credentials

• Report that runtime validation was skipped and why.

Stage 8: Response Contract

Always return:

• Generation path used (script or manual ) and reason.

• Produced file(s) and key plugin choices.

• Validation results (or explicit skip reason with fallback checks performed).

• Assumptions, open risks, and what to customize next.

Done Criteria

This skill execution is complete only when all are true:

• Trigger fit was explicitly confirmed.

• Clarification questionnaire captured required fields or documented assumptions.

• Script-vs-manual decision was explicit and justified.

• Plugin lookup used deterministic chain: Context7 -> official docs -> web (as needed).

• Commands were provided with cwd-safe examples.

• Fallback behavior was applied for any missing tools/docs/network constraints.

• Validation was executed (or skipped with explicit reason and fallback static checks).

• Final response included generated artifacts, validation outcome, and next customization points.

Local Resources

• Script: scripts/generate_config.py

• Templates: examples/*.conf

• Parsers baseline: examples/parsers.conf

• Sample output directory: output/

Use these resources first before introducing new structure.