Which models do you support?

Any open-weight model that runs on Ollama, vLLM, or llama.cpp. Common choices: Llama 3, Mistral, Mixtral, Phi-3, Gemma 2, and Qwen 2. Model selection is based on your performance requirements and hardware constraints.

What hardware do I need?

Minimum for useful inference: a GPU with 8GB VRAM runs 7B models at Q4 quantization. For production serving multiple concurrent users: 24GB VRAM or more. We assess your hardware and recommend the model tier that fits.

Can you set up an OpenAI-compatible API endpoint?

Yes. Both Ollama and vLLM expose OpenAI-compatible endpoints. Your existing applications that call the OpenAI API can be pointed at your local server with a single configuration change.

How do you keep the model updated?

We document a model update procedure as part of every deployment. Updates are tested in a staging environment before production. You own the update process -- we train your team on it before we close.

Local LLM Deployment -- Ollama, vLLM & On-premise Inference

Inference that stays on your hardware and never touches a third-party API. Ollama, vLLM, and llama.cpp deployments on your servers -- full data residency, zero egress cost for inference, and no per-token billing.

What we deploy and operate

Hardware assessment: GPU selection, VRAM planning, and quantization strategy per model family
Ollama deployments for developer-friendly local inference with model management
vLLM for high-throughput production serving with batching and continuous streaming
llama.cpp for CPU-only or memory-constrained environments
API proxy layer: OpenAI-compatible endpoint so your existing applications require no code changes
Model management: version control, hot-swap capability, and rollback procedures

Suitable for healthcare, legal, and financial workloads where data cannot leave your environment.

Your inference stays on your hardware, with full data residency, no per-token billing, and nothing ever reaching a third-party API.

Why teams choose Cobham for local LLM deployment

No application rewrites. An OpenAI-compatible proxy layer means existing apps move off a hosted API with no code changes.
Hardware sized to the model. GPU selection, VRAM planning, and quantization chosen per model family, not guessed.
Built for regulated data. Suitable for healthcare, legal, and financial workloads, with version control and rollback included.

How we engage

Principal-led. Fixed-scope.

Every Local Llm Deployment engagement is scoped before the contract is signed. Two principals. No subcontractors. No account managers between you and the work.

Discovery call

30 minutes. We map your current state, identify the constraints, and tell you what we would do first. No obligation and no invoice.

Scoped brief

A written brief defining deliverables, duration, and a fixed price. If we cannot define the output, we do not take the engagement.

Execution & handoff

Principal-delivered work with a documented handoff. Runbooks, IaC, and access handed to your team at close. You own everything.

Common questions

Frequently asked.

Which models do you support?: Any open-weight model that runs on Ollama, vLLM, or llama.cpp. Common choices: Llama 3, Mistral, Mixtral, Phi-3, Gemma 2, and Qwen 2. Model selection is based on your performance requirements and hardware constraints.
What hardware do I need?: Minimum for useful inference: a GPU with 8GB VRAM runs 7B models at Q4 quantization. For production serving multiple concurrent users: 24GB VRAM or more. We assess your hardware and recommend the model tier that fits.
Can you set up an OpenAI-compatible API endpoint?: Yes. Both Ollama and vLLM expose OpenAI-compatible endpoints. Your existing applications that call the OpenAI API can be pointed at your local server with a single configuration change.
How do you keep the model updated?: We document a model update procedure as part of every deployment. Updates are tested in a staging environment before production. You own the update process -- we train your team on it before we close.

Related services

Other categories we operate.

Cloud

Local Llm Deployment

Talk to a principal.

Schedule a 30-minute call. We map your current state, identify the highest-leverage next step, and give you a written scope before you sign anything.

Schedule a call →

Direct to a principal · Response under four hours

Local Llm Deployment.