ELK Stack Configuration for Log Aggregation

Description

You are an ELK Stack expert. Generate production configurations for centralized log aggregation.

## Input Parameters
- **Deployment**: [DEPLOY] (Docker Compose / Kubernetes / Elastic Cloud)
- **Log Sources**: [SOURCES] (application logs, nginx, system logs, audit logs)
- **Retention Period**: [RETENTION] days
- **Expected Daily Volume**: [VOLUME] GB/day
- **Authentication**: [AUTH] (basic / SAML / LDAP)

## Generate
1. **Elasticsearch** (`elasticsearch.yml`):
   - Cluster settings (node roles, discovery, memory)
   - Index lifecycle management (ILM) policy: hot -> warm -> cold -> delete
   - Index templates with proper mappings and sharding strategy
   - Security: TLS between nodes, authentication enabled
   - Snapshot repository for backups
2. **Logstash** (`logstash.conf`):
   - Input: Beats, syslog, HTTP
   - Filter pipelines per log source:
     - Grok patterns for nginx, application logs
     - Date parsing and timezone handling
     - GeoIP enrichment for access logs
     - Mutate for field normalization
     - Drop debug logs in production
   - Output: Elasticsearch with index routing by source
   - Dead letter queue for failed events
3. **Filebeat** configuration for each log source.
4. **Kibana** (`kibana.yml`):
   - Saved searches for common queries
   - Dashboard definitions for: error rate, response times, top errors, geographic distribution
   - Index patterns with field formatting
   - Alert rules for error spikes
5. **Curator** or ILM for index management.

## Output Format
Return all configuration files with directory structure. Include docker-compose.yml if Docker deployment. Add capacity planning notes based on volume.

Free to copy and use. Compatible with Claude 4 Opus, GPT-5, Gemini 2.5 Pro.

Usage Instructions

Tags

Compatible AI Models

Version History

Related Prompts