Build an automated prompt optimization pipeline using DSPy with signature-based programming, teleprompters, and evaluation.
You are a prompt engineering specialist. Build an automated prompt optimization system using DSPy. ## Configuration - **Task**: [TASK e.g. question answering / summarization / code generation / classification] - **LLM**: [LLM e.g. Claude 4 Sonnet / GPT-5 / Llama 4] - **Training Examples**: [NUM_EXAMPLES e.g. 50 labeled examples] - **Metric**: [METRIC e.g. accuracy / F1 / ROUGE-L / human preference] - **Budget**: [BUDGET e.g. 1000 LLM calls for optimization] ## Pipeline Components ### 1. DSPy Module Definition (`modules/`) - Define Signatures for [TASK]: - Input fields with descriptions - Output fields with constraints - Implement DSPy Modules: - ChainOfThought for reasoning tasks - ReAct for tool-using tasks - ProgramOfThought for code-generation tasks - MultiChainComparison for ensemble approaches - Compose modules into a pipeline with assertions ### 2. Teleprompter Configuration (`optimization/`) - Configure optimization strategies: - BootstrapFewShot: auto-select best demonstrations - BootstrapFewShotWithRandomSearch: explore prompt variations - MIPRO: instruction optimization with Bayesian search - BootstrapFinetune: distill into smaller models - Set [BUDGET] constraint for optimization - Multi-objective optimization: quality + cost + latency ### 3. Evaluation Framework (`evaluation/`) - Load and split [NUM_EXAMPLES] into train/val/test - Implement [METRIC] as DSPy metric function - Add secondary metrics for analysis - Bootstrap confidence intervals - Compare optimized vs baseline prompts - Cost analysis: tokens used, API calls, total spend ### 4. Prompt Analysis (`analysis/`) - Extract optimized instructions and demonstrations - Analyze which examples were selected and why - Visualize prompt component contributions - A/B test framework for deployment - Export optimized prompts as reusable templates ### 5. Deployment (`deploy/`) - Serialize optimized DSPy programs - FastAPI wrapper for inference - Caching layer for repeated queries - Fallback to unoptimized prompt on errors Provide complete code with example datasets for [TASK] and comparison benchmarks.
Free to copy and use. Compatible with Claude 4 Opus, Claude 4 Sonnet, GPT-5, Gemini 2.5 Pro.
Define your task signatures and prepare labeled examples. Start with BootstrapFewShot for quick optimization, then try MIPRO for instruction tuning. Compare results on held-out test set.
Initial release
Sign in and download this prompt to leave a review.