Data Expert
data analysis initial exploration
When reviewing or writing code, apply these guidelines:
-
Begin analysis with data exploration and summary statistics.
-
Implement data quality checks at the beginning of analysis.
-
Handle missing data appropriately (imputation, removal, or flagging).
data fetching rules for server components
When reviewing or writing code, apply these guidelines:
- For data fetching in server components (in .tsx files): tsx async function getData() { const res = await fetch('https://api.example.com/data', { next: { revalidate: 3600 } }) if (!res.ok) throw new Error('Failed to fetch data') return res.json() } export default async function Page() { const data = await getData() // Render component using data }
data pipeline management with dvc
When reviewing or writing code, apply these guidelines:
- Data Pipeline Management: Employ scripts or tools like dvc to manage data preprocessing and ensure reproducibility.
data synchronization rules
When reviewing or writing code, apply these guidelines:
-
Implement Data Synchronization:
-
Create an efficient system for keeping the region grid data synchronized between the JavaScript UI and the WASM simulation. This might involve: a. Implementing periodic updates at set intervals. b. Creating an event-driven synchronization system that updates when changes occur. c. Optimizing large data transfers to maintain smooth performance, possibly using typed arrays or other efficient data structures. d. Implementing a queuing system for updates to prevent overwhelming the simulation with rapid changes.
data tracking and charts rule
When reviewing or writing code, apply these guidelines:
- There should be a chart page that tracks just about everything that can be tracked in the game.
data validation with pydantic
When reviewing or writing code, apply these guidelines:
- Data Validation: Use Pydantic models for rigorous
Consolidated Skills
This expert skill consolidates 1 individual skills:
- data-expert
Iron Laws
-
ALWAYS validate all external data at system boundaries using a schema validator (Zod, Pydantic, Joi) — never trust API responses, user input, or file contents without validation.
-
NEVER load entire large datasets into memory — always stream, paginate, or batch-process data beyond a few thousand records to prevent memory spikes and timeouts.
-
ALWAYS sanitize data before using it in downstream operations — HTML, SQL, and shell-injected content must be stripped or escaped before processing or storage.
-
NEVER use string manipulation (regex, split, replace) as a primary parser for structured formats — use purpose-built parsers (JSON.parse, csv-parse, xml2js) for reliable type-safe results.
-
ALWAYS make data transformation functions pure and idempotent — a function that mutates external state or produces different results for the same input cannot be safely tested or reused.
Anti-Patterns
Anti-Pattern Why It Fails Correct Approach
Trusting API responses without validation API schemas change silently; unvalidated data causes downstream type errors Validate all responses with Zod/Pydantic schemas at the API boundary
fs.readFileSync on large CSV/JSON files Loads entire file into memory; crashes on files > available RAM Use streaming parsers (csv-parse/stream, JSONStream) with backpressure
Regex for parsing HTML or XML HTML/XML structure is not regular; regex breaks on nested tags and attributes Use proper DOM/XML parsers (cheerio, xml2js, DOMParser)
Mutating input objects in transformations Caller still holds a reference to the mutated object; causes ghost bugs Return new objects ({ ...input, newField } ) instead of mutating
Logging full request/response bodies with PII PII ends up in log aggregators readable by non-authorized users Redact PII fields before logging; log schemas and IDs only
Memory Protocol (MANDATORY)
Before starting:
cat .claude/context/memory/learnings.md
After completing: Record any new patterns or exceptions discovered.
ASSUME INTERRUPTION: Your context may reset. If it's not in memory, it didn't happen.