unsubbed.co

LLM Harbor

LLM Harbor is a TypeScript-based application that provides containerized LLM toolkit. Run LLM backends.

Overview

One command brings a complete pre-wired LLM stack with hundreds of services to explore. https://github.com/user-attachments/assets/8a7705e1-6f0e-4374-8784-62b95816aebc The project has 3K+ GitHub stars and is licensed under Apache-2.0.

Getting Started

Source: GitHub README

harbor logs llamacpp harbor exec llamacpp ./scripts/llama-bench —help harbor shell vllm

Normalized Features

Source: tool-features-normalized.json

docker, docker compose, plugins, rest api.

Features

Integrations & APIs

  • Plugin / Extension System
  • REST API