Groq Events & Async Patterns

Overview

Build event-driven architectures around Groq's ultra-fast LLM inference API. Groq does not provide native webhooks, but its sub-second response times at api.groq.com enable unique patterns: real-time streaming, batch processing with callbacks, and event-driven pipelines where Groq acts as the processing engine within your webhook infrastructure.

Prerequisites

• Groq API key stored in GROQ_API_KEY environment variable

• Groq SDK installed (npm install groq-sdk or pip install groq )

• Queue system for batch processing (BullMQ, Celery)

• Understanding of Groq model options (llama, mixtral, gemma)

Event Patterns

Pattern Trigger Use Case

Streaming SSE Client request Real-time chat responses

Batch completion callback Queue job finishes Document processing pipeline

Webhook processor Incoming webhook Process events with Groq LLM

Health monitor Scheduled check API availability tracking

Instructions

Step 1: Real-Time Streaming with SSE

import Groq from "groq-sdk";

const groq = new Groq({ apiKey: process.env.GROQ_API_KEY! });

// Express SSE endpoint app.post("/api/chat/stream", async (req, res) => { const { messages, model } = req.body;

res.writeHead(200, { # HTTP 200 OK "Content-Type": "text/event-stream", "Cache-Control": "no-cache", "Connection": "keep-alive", });

const stream = await groq.chat.completions.create({ model: model || "llama-3.3-70b-versatile", messages, stream: true, max_tokens: 2048, # 2048: 2 KB });

for await (const chunk of stream) { const content = chunk.choices[0]?.delta?.content; if (content) { res.write(data: ${JSON.stringify({ content })}\n\n); } }

res.write("data: [DONE]\n\n"); res.end(); });

Step 2: Batch Processing with Callbacks

import { Queue, Worker } from "bullmq";

const groqQueue = new Queue("groq-batch");

// Queue a batch of prompts with callback webhook async function queueBatch(prompts: string[], callbackUrl: string) { const batchId = crypto.randomUUID();

for (const [index, prompt] of prompts.entries()) { await groqQueue.add("inference", { batchId, index, prompt, callbackUrl, totalItems: prompts.length, }); }

return batchId; }

const worker = new Worker("groq-batch", async (job) => { const { prompt, callbackUrl, batchId, index, totalItems } = job.data;

const completion = await groq.chat.completions.create({ model: "llama-3.3-70b-versatile", messages: [{ role: "user", content: prompt }], });

const result = { batchId, index, content: completion.choices[0].message.content, usage: completion.usage, model: completion.model, processingTime: completion.usage?.total_time, };

// Fire callback webhook on completion await fetch(callbackUrl, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ event: "groq.batch_item.completed", data: result, }), });

return result; });

Step 3: Webhook Event Processor

// Use Groq to process incoming webhook events with LLM async function processWebhookWithGroq(event: any) { const completion = await groq.chat.completions.create({ model: "llama-3.1-8b-instant", messages: [ { role: "system", content: "Classify this event and extract key information. Respond with JSON.", }, { role: "user", content: JSON.stringify(event), }, ], response_format: { type: "json_object" }, temperature: 0, });

return JSON.parse(completion.choices[0].message.content!); }

Step 4: Monitor API Health

async function checkGroqHealth(): Promise<boolean> { try { const start = Date.now(); await groq.chat.completions.create({ model: "llama-3.1-8b-instant", messages: [{ role: "user", content: "ping" }], max_tokens: 1, }); const latency = Date.now() - start; console.log(Groq health: OK (${latency}ms)); return true; } catch (error) { console.error("Groq health check failed:", error); return false; } }

Error Handling

Issue Cause Solution

Rate limited (429) Too many requests Implement exponential backoff, use queue

Model unavailable Service capacity Fall back to smaller model variant

Streaming disconnect Network timeout Implement reconnection with last token

JSON parse error Malformed response Use response_format and validate output

Examples

Python Async Batch

import asyncio from groq import AsyncGroq

client = AsyncGroq(api_key=os.environ["GROQ_API_KEY"])

async def process_batch(prompts: list[str]): tasks = [ client.chat.completions.create( model="llama-3.3-70b-versatile", messages=[{"role": "user", "content": p}], ) for p in prompts ] return await asyncio.gather(*tasks)

Resources

• Groq API Documentation

• Groq Models

• Groq Rate Limits

Next Steps

For deployment setup, see groq-deploy-integration .

Output

• Configuration files or code changes applied to the project

• Validation report confirming correct implementation

• Summary of changes made and their rationale