Label Studio Setup

Skill Profile

(Select at least one profile to enable specific modules)

• DevOps

• Backend

• Frontend

• AI-RAG

• Security Critical

Overview

Label Studio is an open-source data labeling platform that provides tools for image, text, audio, and video annotation. This skill covers Label Studio installation, project setup, data import/export, labeling interface customization, user management, quality control, ML backend integration, API usage, backup and migration, and production deployment.

Why This Matters

• Data Quality: High-quality labeled data for ML models

• Efficiency: Streamlined annotation workflows

• Collaboration: Team-based annotation with review

• Automation: ML backend for pre-annotation

• Flexibility: Custom annotation interfaces

Core Concepts & Rules

1. Core Principles

• Follow established patterns and conventions

• Maintain consistency across codebase

• Document decisions and trade-offs

2. Implementation Guidelines

• Start with the simplest viable solution

• Iterate based on feedback and requirements

• Test thoroughly before deployment

Inputs / Outputs / Contracts

Skill Composition

• Depends on: None

• Compatible with: None

• Conflicts with: None

• Related Skills: None

Quick Start / Implementation Example

• Review requirements and constraints

• Set up development environment

• Implement core functionality following patterns

• Write tests for critical paths

• Run tests and fix issues

• Document any deviations or decisions

Example implementation following best practices

def example_function(): # Your implementation here pass

Assumptions

• Docker is available

• Sufficient storage for media files

• Network connectivity for team access

• ML backend is optional

Compatibility

• Python: Full support

• Docker: Full support

• PostgreSQL: Full support

• Redis: Full support

• Flask: Partial support

Test Scenario Matrix (QA Strategy)

Type Focus Area Required Scenarios / Mocks

Unit Core Logic Must cover primary logic and at least 3 edge/error cases. Target minimum 80% coverage

Integration DB / API All external API calls or database connections must be mocked during unit tests

E2E User Journey Critical user flows to test

Performance Latency / Load Benchmark requirements

Security Vuln / Auth SAST/DAST or dependency audit

Frontend UX / A11y Accessibility checklist (WCAG), Performance Budget (Lighthouse score)

Technical Guardrails & Security Threat Model

1. Security & Privacy (Threat Model)

• Top Threats: Injection attacks, authentication bypass, data exposure

• Data Handling: Sanitize all user inputs to prevent Injection attacks. Never log raw PII

• Secrets Management: No hardcoded API keys. Use Env Vars/Secrets Manager

• Authorization: Validate user permissions before state changes

2. Performance & Resources

• Execution Efficiency: Consider time complexity for algorithms

• Memory Management: Use streams/pagination for large data

• Resource Cleanup: Close DB connections/file handlers in finally blocks

3. Architecture & Scalability

• Design Pattern: Follow SOLID principles, use Dependency Injection

• Modularity: Decouple logic from UI/Frameworks

4. Observability & Reliability

• Logging Standards: Structured JSON, include trace IDs request_id

• Metrics: Track error_rate , latency , queue_depth

• Error Handling: Standardized error codes, no bare except

• Observability Artifacts:

• Log Fields: timestamp, level, message, request_id

• Metrics: request_count, error_count, response_time

• Dashboards/Alerts: High Error Rate > 5%

Agent Directives & Error Recovery

(ข้อกำหนดสำหรับ AI Agent ในการคิดและแก้ปัญหาเมื่อเกิดข้อผิดพลาด)

• Thinking Process: Analyze root cause before fixing. Do not brute-force.

• Fallback Strategy: Stop after 3 failed test attempts. Output root cause and ask for human intervention/clarification.

• Self-Review: Check against Guardrails & Anti-patterns before finalizing.

• Output Constraints: Output ONLY the modified code block. Do not explain unless asked.

Definition of Done (DoD) Checklist

• Tests passed + coverage met

• Lint/Typecheck passed

• Logging/Metrics/Trace implemented

• Security checks passed

• Documentation/Changelog updated

• Accessibility/Performance requirements met (if frontend)

Anti-patterns / Pitfalls

• ⛔ Don't: Log PII, catch-all exception, N+1 queries

• ⚠️ Watch out for: Common symptoms and quick fixes

• 💡 Instead: Use proper error handling, pagination, and logging

Reference Links & Examples

• Internal documentation and examples

• Official documentation and best practices

• Community resources and discussions

Versioning & Changelog

• Version: 1.0.0

• Changelog:

• 2026-02-22: Initial version with complete template structure