GrepAI Embeddings with LM Studio

This skill covers using LM Studio as the embedding provider for GrepAI, offering a user-friendly GUI for managing local models.

When to Use This Skill

• Want local embeddings with a graphical interface

• Already using LM Studio for other AI tasks

• Prefer visual model management over CLI

• Need to easily switch between models

What is LM Studio?

LM Studio is a desktop application for running local LLMs with:

• 🖥️ Graphical user interface

• 📦 Easy model downloading

• 🔌 OpenAI-compatible API

• 🔒 100% private, local processing

Prerequisites

• Download LM Studio from lmstudio.ai

• Install and launch the application

• Download an embedding model

Installation

Step 1: Download LM Studio

Visit lmstudio.ai and download for your platform:

• macOS (Intel or Apple Silicon)

• Windows

• Linux

Step 2: Launch and Download a Model

• Open LM Studio

• Go to the Search tab

• Search for an embedding model:

• nomic-embed-text-v1.5

• bge-small-en-v1.5

• bge-large-en-v1.5

• Click Download

Step 3: Start the Local Server

• Go to the Local Server tab

• Select your embedding model

• Click Start Server

• Note the endpoint (default: http://localhost:1234 )

Configuration

Basic Configuration

.grepai/config.yaml

embedder: provider: lmstudio model: nomic-embed-text-v1.5 endpoint: http://localhost:1234

With Custom Port

embedder: provider: lmstudio model: nomic-embed-text-v1.5 endpoint: http://localhost:8080

With Explicit Dimensions

embedder: provider: lmstudio model: nomic-embed-text-v1.5 endpoint: http://localhost:1234 dimensions: 768

Available Models

nomic-embed-text-v1.5 (Recommended)

Property Value

Dimensions 768

Size ~260 MB

Quality Excellent

Speed Fast

embedder: provider: lmstudio model: nomic-embed-text-v1.5

bge-small-en-v1.5

Property Value

Dimensions 384

Size ~130 MB

Quality Good

Speed Very fast

Best for: Smaller codebases, faster indexing.

embedder: provider: lmstudio model: bge-small-en-v1.5 dimensions: 384

bge-large-en-v1.5

Property Value

Dimensions 1024

Size ~1.3 GB

Quality Very high

Speed Slower

Best for: Maximum accuracy.

embedder: provider: lmstudio model: bge-large-en-v1.5 dimensions: 1024

Model Comparison

Model Dims Size Speed Quality

bge-small-en-v1.5

384 130MB ⚡⚡⚡ ⭐⭐⭐

nomic-embed-text-v1.5

768 260MB ⚡⚡ ⭐⭐⭐⭐

bge-large-en-v1.5

1024 1.3GB ⚡ ⭐⭐⭐⭐⭐

LM Studio Server Setup

Starting the Server

• Open LM Studio

• Navigate to Local Server tab (left sidebar)

• Select an embedding model from the dropdown

• Configure settings:

• Port: 1234 (default)

• Enable Embedding Endpoint

• Click Start Server

Server Status

Look for the green indicator showing the server is running.

Verifying the Server

Check server is responding

curl http://localhost:1234/v1/models

Test embedding

curl http://localhost:1234/v1/embeddings
-H "Content-Type: application/json"
-d '{ "model": "nomic-embed-text-v1.5", "input": "function authenticate(user)" }'

LM Studio Settings

Recommended Settings

In LM Studio's Local Server tab:

Setting Recommended Value

Port 1234

Enable CORS Yes

Context Length Auto

GPU Layers Max (for speed)

GPU Acceleration

LM Studio automatically uses:

• macOS: Metal (Apple Silicon)

• Windows/Linux: CUDA (NVIDIA)

Adjust GPU layers in settings for memory/speed balance.

Running LM Studio Headless

For server environments, LM Studio supports CLI mode:

Start server without GUI (check LM Studio docs for exact syntax)

lmstudio server start --model nomic-embed-text-v1.5 --port 1234

Common Issues

❌ Problem: Connection refused ✅ Solution: Ensure LM Studio server is running:

• Open LM Studio

• Go to Local Server tab

• Click Start Server

❌ Problem: Model not found ✅ Solution:

• Download the model in LM Studio's Search tab

• Select it in the Local Server dropdown

❌ Problem: Slow embedding generation ✅ Solutions:

• Enable GPU acceleration in LM Studio settings

• Use a smaller model (bge-small-en-v1.5)

• Close other GPU-intensive applications

❌ Problem: Port already in use ✅ Solution: Change port in LM Studio settings:

embedder: endpoint: http://localhost:8080 # Different port

❌ Problem: LM Studio closes and server stops ✅ Solution: Keep LM Studio running in the background, or consider using Ollama which runs as a system service

LM Studio vs Ollama

Feature LM Studio Ollama

GUI ✅ Yes ❌ CLI only

System service ❌ App must run ✅ Background service

Model management ✅ Visual ✅ CLI

Ease of use ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐

Server reliability ⭐⭐⭐ ⭐⭐⭐⭐⭐

Recommendation: Use LM Studio if you prefer a GUI, Ollama for always-on background service.

Migrating from LM Studio to Ollama

If you need a more reliable background service:

• Install Ollama:

brew install ollama ollama serve & ollama pull nomic-embed-text

• Update config:

embedder: provider: ollama model: nomic-embed-text endpoint: http://localhost:11434

• Re-index:

rm .grepai/index.gob grepai watch

Best Practices

• Keep LM Studio running: Server stops when app closes

• Use recommended model: nomic-embed-text-v1.5 for best balance

• Enable GPU: Faster embeddings with hardware acceleration

• Check server before indexing: Ensure green status indicator

• Consider Ollama for production: More reliable as background service

Output Format

Successful LM Studio configuration:

✅ LM Studio Embedding Provider Configured

Provider: LM Studio Model: nomic-embed-text-v1.5 Endpoint: http://localhost:1234 Dimensions: 768 (auto-detected) Status: Connected

Note: Keep LM Studio running for embeddings to work.