Running a Local Model Alongside Frontier Models in Production

The phrase 'local model' is used loosely in the AI industry and it is worth being precise, because the distinction has real consequences for how you architect your workflows.

Model hosting is the question of where the model weights run and who operates the inference server. When you run Ollama on a machine you control, you are hosting the model. Your hardware, your inference, your data boundary.

Platform hosting is the question of where the workflow orchestration platform runs. DAVE is fully managed and hosted by voolama LLC. There is no self-hosted, on-premises, or customer-operated version of DAVE. You do not install DAVE on a VPS, a Kubernetes cluster, or your own infrastructure. The platform runs on voolama's managed infrastructure.

These two things are independent. DAVE (the platform) is managed by voolama. The models your agents call can be anywhere: a commercial API, a cloud-hosted endpoint, or an Ollama instance running on hardware you control. When a DAVE agent calls your Ollama instance, the call goes from voolama's managed infrastructure to your Ollama endpoint over HTTPS. The platform is managed. The model is local to you.

This distinction matters for three practical reasons:

• Data routing: If you run Ollama locally, the prompt and response travel between voolama's infrastructure and your Ollama endpoint. They do not travel to a commercial model provider's API. For workloads where sending data to a third-party API is a concern, this is the relevant boundary to understand.
• Reliability: A local Ollama instance is only as reliable as the hardware and network you operate. Commercial API providers have SLAs, redundancy, and global infrastructure. Your Ollama instance does not, unless you build that yourself.
• Latency: Calls to a local Ollama instance traverse your network and voolama's infrastructure. Depending on your network topology, this may be faster or slower than a commercial API call. Test with your actual workload before committing to a production architecture.

DAVE supports seven provider types. One of them is Local (Ollama), with a default base URI of http://localhost:11434/v1. To connect your Ollama instance, navigate to Admin, then API Management, then Providers and create a new provider record.

In the create form:

1. Set Name to something descriptive, such as "Ollama Production" or "Ollama Dev". The name appears in agent configuration dropdowns, so clarity here saves confusion later.
2. Set Provider Type to Local (Ollama). The Base URI field auto-fills with http://localhost:11434/v1.
3. Update the Base URI to the actual network address of your Ollama instance. The default assumes Ollama is running on the same host as the caller, which is not the case when DAVE's managed infrastructure is calling your endpoint. Replace it with the reachable address of your Ollama server, for example https://ollama.yourcompany.com/v1 or the IP and port your Ollama instance listens on. The endpoint must be reachable from voolama's infrastructure over HTTPS.
4. The API Key field is optional for Ollama. If your Ollama instance is not protected by an API key, leave it blank. If you have configured Ollama to require authentication, enter the key here. It will be stored encrypted using AES-256-GCM authenticated encryption and never returned through the API once stored.
5. Click Fetch Models to retrieve the list of models currently loaded in your Ollama instance. Check the models you want to make available to agents. Click Create Provider to save.

After saving, use Fetch Models on the provider card to confirm the connection is live. A successful fetch means DAVE can reach your Ollama endpoint and authenticate. If the fetch fails, check the base URI, confirm your Ollama instance is running and reachable, and verify any firewall or network rules that might block the connection from voolama's infrastructure.

The Providers page is restricted to the Admin role. Users with Create, Curate, Use, or Reporting roles cannot view, create, edit, or delete providers.

There is no limit in DAVE to the number of provider records you configure, and no requirement that all agents use the same provider. You can have an OpenAI provider, an Anthropic provider, a Google AI provider, and an Ollama provider all active in the same workspace simultaneously. Each is a separate record at Admin, then API Management, then Providers, each with its own API key, base URI, and model catalog.

Agents reference providers individually. When you configure an agent, you select the provider and the model from that provider's catalog. Different agents in the same workflow can reference different providers. A workflow that calls three agents can route each agent call to a different model on a different provider, all in the same workflow instance.

This is the architecture that makes mixed-model workflows possible:

• An Agent Interaction node calls a frontier model (OpenAI, Anthropic, Google AI) for a task that benefits from maximum capability, such as complex reasoning, nuanced writing, or multi-step analysis.
• A different Agent Interaction node in the same workflow calls an Ollama-hosted open-weight model for a task that is well-defined, repetitive, and does not require frontier capability, such as classification, extraction, or formatting.
• A third Agent Interaction node calls whichever model is most cost-effective for its specific task.

The workflow graph does not change when you change which model an agent uses. The provider and model selection live in the agent configuration, not in the workflow structure. Swapping a frontier model for a local model on a specific agent means editing the agent, not rebuilding the workflow.

For teams running multiple providers, each provider bills your account directly at the provider's published rate. DAVE does not proxy or mark up AI usage. Your Ollama instance has no per-call API cost (you bear the infrastructure cost of running it). Your commercial provider calls are billed by the provider. The two cost structures are independent.

The decision of which model to use for a given task is not a one-time architecture choice. It is a per-agent, per-task question that you can revisit and change as models improve and your understanding of your workload deepens. The following criteria are a starting point, not a fixed rule.

Tasks that suit a local open-weight model

• Well-defined extraction: Pulling structured fields from a known document format. The task is deterministic enough that a capable open-weight model handles it reliably, and the volume may be high enough that the per-call cost of a frontier API matters.
• Classification and routing: Assigning a label from a fixed set, deciding which branch a workflow should take, or scoring a record against a rubric. These tasks are often within the capability of smaller models.
• Formatting and normalization: Converting unstructured text into a defined schema, normalizing date formats, or cleaning records. High volume, low complexity.
• Data-sensitive workloads: Tasks where sending the prompt to a third-party commercial API is a concern and you prefer the data to stay within your own infrastructure boundary.

Tasks that suit a frontier model

• Complex reasoning: Multi-step analysis, synthesis across long documents, or tasks that require the model to hold and reason about a large amount of context simultaneously.
• Nuanced generation: Writing that requires tone judgment, audience awareness, or creative quality that smaller models do not reliably produce.
• Novel or ambiguous inputs: Tasks where the input format or content is unpredictable and the model needs broad capability to handle edge cases gracefully.
• Agent review and quality gates: Using an Agent Review node to evaluate the output of another agent. The reviewing model often needs to be at least as capable as the model being reviewed.

The practical approach

Start with a frontier model for every agent. Once the workflow is stable and you understand what each agent is actually doing, identify the agents whose tasks fit the criteria above and test them against an Ollama-hosted model. Compare output quality on a representative sample of real inputs. If the quality is acceptable, switch the agent to the local provider. If it is not, keep the frontier model. The workflow graph does not change either way.

Adding a local Ollama instance to a production workflow introduces operational responsibilities that commercial API providers handle for you. These are not reasons to avoid local models, but they are things to plan for before you depend on them in a production workflow.

Availability and uptime

If your Ollama instance goes offline, every workflow instance that reaches an agent configured to use it will fail or pause at that node. DAVE's managed infrastructure is not responsible for the availability of your Ollama endpoint. Build your Ollama deployment with the same availability requirements you would apply to any production dependency: redundancy, monitoring, and a recovery plan.

Model updates

When you update a model in your Ollama instance (pulling a new version of a model), the behavior of agents using that model may change. Unlike commercial API providers, which version their models explicitly (for example, gpt-4o-2024-08-06), Ollama model tags may resolve to different weights over time. If consistency of model behavior matters for your workflow, pin to a specific model version tag in Ollama and update deliberately.

After updating models in your Ollama instance, click Fetch Models on the provider card in DAVE to refresh the model catalog. If a model's ID changes between versions, update the agent configuration to reference the new ID.

Key rotation

If your Ollama instance requires an API key for authentication, rotate it by editing the provider record at Admin, then API Management, then Providers. Because all agents that reference this provider share the same provider record, updating the key in one place propagates to every agent on the next invocation. No workflow graphs change.

Network reachability

Your Ollama endpoint must be reachable from voolama's managed infrastructure. This typically means exposing the endpoint over HTTPS with a valid certificate, and configuring any firewall or network rules to permit inbound connections from voolama's infrastructure. The default Ollama base URI (http://localhost:11434/v1) is a local loopback address and is not reachable from external infrastructure. You must configure a reachable endpoint and update the Base URI field in the provider record accordingly.