Skip to main content

LM Evaluation Harness

Skills/MLOps
MLOps

LM Evaluation Harness

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Enabled by defaultBuilt In
CLI install commandaegis skills install lm-evaluation-harness
Install guide
Overview

Bundled with the packaged Aegis CLI as a built-in procedural skill.

Already ships inside the packaged Aegis bundle. Use `aegis skills install lm-evaluation-harness` only when you want an explicit local materialization record.

Continue browsing

Also in MLOps

Model, inference, evaluation, and retrieval workflows kept explicit, reproducible, and operator-owned.

View full section
AudioCraft

PyTorch library for audio generation including text-to-music (MusicGen) and text-to-sound (AudioGen). Use when you need to generate music from text descriptions, create sound effects, or perform melody-conditioned music generation.

audiocraftView skill
Axolotl

Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support

axolotlView skill
CLIP

OpenAI's model connecting vision and language. Enables zero-shot image classification, image-text matching, and cross-modal retrieval. Trained on 400M image-text pairs. Use for image search, content moderation, or vision-language tasks without fine-tuning. Best for general-purpose image understanding.

clipView skill