B3IQ Blog · Knowledge · 8 min read
Customer-Owned AI Infrastructure: A Buyer’s Guide to Migrating Existing ML Inference
What changes when your application moves to GPUs you own, from model discovery and API keys to capacity, testing, and recovery.
B3IQ Team
You can move supported inference calls to customer-owned GPUs when your machines serve a suitable model and your application can use the documented API. Validate model behavior, capacity, and recovery before switching production traffic.
Fig. 1: Migrating Existing ML Inference — white text over a blurred green and blue landscape.
TL;DR
- You can evaluate moving existing inference calls to customer-owned B3IQ machines if you own or plan to buy suitable bare-metal hardware and your application can use the documented inference interface.
- B3IQ does not charge per request for inference on owned machines. You still pay for the machine, hosting, and other operating costs.
- B3IQ can host customer-owned machines and provide physical operations. You remain responsible for your application, workload fit, and sufficient capacity. If your eligible machines cannot serve a request, the owner-scoped gateway does not send it to a public pool or third-party provider.
What B3IQ actually supports today
B3IQ’s documented hosted inference path uses an owner-scoped gateway to route requests to customer-owned machines. B3IQ also describes buying a machine for hosting in Oregon or taking delivery to operate it at your location. Confirm which inference access and operating arrangements apply to the machine you choose.
For the hosted gateway, the SDK quickstart documents API-key authentication, client.models.list() for model discovery, and OpenAI-compatible chat completions, including streaming. Your application must request a model ID returned for your account. Check any other modality against its specific endpoint and runtime documentation before treating it as supported for your deployment.
The gateway keeps inference requests within your owned fleet. According to the private-fleet guide, a machine must be online, healthy, in Bare metal mode, and serving the requested model. If no eligible owned machine can serve a request, the request fails. B3IQ does not route it to a shared public pool or automatically fall back to a third-party provider.
Eligible machines can enter Earn mode. While in that mode, your owner-scoped inference keys cannot reach them, and SSH is unavailable until they return to Bare metal mode. The private-fleet guide says B3IQ schedules mode changes with the owner. Bare-metal SSH access also lets you run your own workloads, but it does not constitute a managed training service. Managed training is planned separately from the ownership offering. Verified confidential execution is also not currently offered.
Is your current inference workload a fit?
B3IQ is a potential fit if your application uses OpenAI-compatible chat completions and you have an owned machine that can serve the required model. Before changing code, use your account key to discover the model IDs available to your deployment, as shown in the SDK quickstart. Check any provider-specific features your application uses, since a compatible API does not guarantee identical model behavior.
Test each call pattern your application depends on. For streaming chat, check that your application handles the returned stream as expected. Verify embeddings against their own endpoint and available models rather than assuming chat support covers them. Model availability, GPU memory, context length, and expected concurrency all affect whether the intended machine can handle your workload. Sustained demand may make owned capacity worth evaluating, while long idle periods or sharp traffic bursts call for a closer capacity assessment.
Plan separately for requests that the owned fleet cannot serve. Under B3IQ’s private-fleet rules, a request fails if no eligible owned machine is healthy and serving its model. B3IQ does not automatically send it to a rented GPU provider, so bursts and outages need a capacity or recovery plan. Assess training separately from this inference migration; B3IQ lists managed training workflows as planned work.
Migration checklist before you cut over
- Audit the calls your application makes. Record each endpoint, model ID, request parameter, streaming requirement, and provider-specific feature. Compare those calls with B3IQ’s documented SDK methods. Changing the base URL does not preserve a model ID or transfer proprietary model weights.
- Confirm capacity and discover models. Check that an owned machine is online, in Bare metal mode, and serving the model you intend to use. With an owner API key, call
client.models.list()and select a returned ID for a model that supports the endpoint your application uses. Discovery reflects the machines your key can reach at that time, including any machine scope set on the key. - Configure credentials outside the application code. Put the hosted gateway key in
B3IQ_API_KEY, and keep it out of source control and literal command-line arguments. Confirm that the key reaches the intended machines before changing application traffic. - Stage the endpoint change. Point a nonproduction client at
https://b3iq.org/v1/apiand use a discovered model ID. Keep the prior provider configuration available for rollback. Neither matching request syntax nor a successful response guarantees matching model behavior. - Test the application’s actual workload. Compare representative outputs for quality, and measure latency, throughput, and concurrency under expected traffic. Check streaming if your application uses it. Test invalid credentials, unavailable models, and the response when no eligible owned machine can serve a request.
- Assign ownership before cutover. Decide who monitors machine and model availability, who investigates failed requests, and who switches traffic back if capacity becomes unavailable. B3IQ does not automatically route those requests to a public GPU pool. Confirm B3IQ’s support and recovery scope alongside your own responsibilities before sending production traffic.
Worked example: swapping the endpoint
The B3IQ SDK quickstart uses the OpenAI Python client with a different base URL and an owner API key. The example below adapts that documentation to discover available model IDs before making a chat request. Install the openai package. In a Bash shell, run the commands below and enter your current b3iq_key_ key at the prompt; the key will not appear as you type.
read -r -s -p "B3IQ API key: " B3IQ_API_KEY
printf '\n'
export B3IQ_API_KEYimport os
from openai import OpenAI
client = OpenAI(
base_url="https://b3iq.org/v1/api",
api_key=os.environ["B3IQ_API_KEY"],
)
available_ids = {model.id for model in client.models.list()}
if not available_ids:
raise RuntimeError("No models are available on your account")
print("Available model IDs:", sorted(available_ids))
model_id = input("Enter an available chat model ID > ").strip()
if model_id not in available_ids:
raise ValueError("Choose an exact model ID from the list")
reply = client.chat.completions.create(
model=model_id,
messages=[{"role": "user", "content": "Hi"}],
)
print(reply.choices[0].message.content)Authenticated discovery lists models your account’s eligible machines currently serve. Select a model that supports your chat request. Older examples may show b3iq_gateway_ keys, but that prefix is retired. Changing the endpoint does not transfer proprietary model weights or guarantee the same behavior or output as your previous provider.
Authentication and endpoint changes to plan for
B3IQ hosted keys can reach every machine you own by default, but you can scope a key to specific machines. The gateway verifies ownership on every request before routing it. Machine scope limits which owned machines a key can reach, so check that the permitted machines serve the models your application needs.
For the hosted gateway at https://b3iq.org/v1/api, load the b3iq_key_ secret at runtime from an environment variable such as B3IQ_API_KEY or a protected file, as shown in the SDK quickstart. Do not put the key in source code or pass it as a literal command-line argument.
The local host-agent runs on the GPU machine at http://127.0.0.1:8831 by default and uses separate b3iq_local_ credentials. That loopback address is reachable from the GPU machine itself; entering it on your laptop addresses your laptop. Access from another machine requires an appropriately configured connection or tunnel. Keep local API credentials and configuration separate from the hosted gateway used in this example.
Owned machines versus rented GPU services
Rented GPUs and customer-owned machines divide operating work differently. A GPU-as-a-service overview describes rental providers as responsible for hardware and facilities, while customers manage the software and models they run. The division of work varies by service; check the specific offering.
- Consideration · Rented GPU service · Self-hosted owned machines · B3IQ-managed hosting
- Control · You select available rental capacity and manage your workload. · You choose the machines and models, and operate the site. · You own the machines and manage the workload. B3IQ handles physical hosting and operations.
- Capacity · On-demand rental can accommodate variable use, while reserved rental capacity typically requires a commitment. · You purchase enough hardware for the capacity you need. · Your requests reach only eligible machines you own. If none can serve the request, it fails rather than falling back to a public pool.
- Costs · Charges depend on the rental arrangement and usage. · You pay for hardware, facilities, and operations. · B3IQ charges no per-request fee for inference on owned machines. Hardware, hosting, and other operating costs remain.
B3IQ’s private-fleet guide documents owner-only routing and the absence of a per-request inference charge. The ownership guide describes B3IQ-managed hosting, while the private-fleet guide explains what happens when no eligible machine can serve a request. Confirm the physical services and charges in a proposed hosting arrangement rather than treating managed hosting as equivalent to operating your own site.
Compare costs over the same operating period and traffic pattern, using the same model and capacity requirements where both options can support them. For ownership, include hardware, hosting, and any power, maintenance, and staffing costs you bear. Enzu’s vendor-authored comparison contrasts dedicated capacity with rentals, but use your workload and quoted terms rather than that comparison to judge cost.
Bare-metal training versus managed training services
You can run your own training jobs on a B3IQ machine in Bare metal mode using SSH access. You remain responsible for installing the training software, managing the job, and checking its results. Bare-metal access does not mean B3IQ operates a managed training service.
B3IQ lists managed training workflows as planned work. Its current documentation does not establish a managed PyTorch or TensorFlow training integration. If training matters to your longer-term plans, assess a managed service when B3IQ publishes its supported scope. Base the current migration decision on the documented inference path and a test of your workload on the intended machine.
Before you move production traffic
- Confirm that the intended machine is online, healthy, and serving the required model in the planned deployment. Use the intended API key to check model discovery. The private-fleet guide explains the routing requirements.
- Run a limited pilot against your application’s requirements. Follow the SDK quickstart to send authenticated requests, then check representative outputs, latency, and error handling.
- Test recovery before production cutover. If an owned machine cannot serve a request, B3IQ does not automatically route it to a public pool. Confirm who handles recovery and what support covers. The gateway documentation explains the routing requirements and what happens when no eligible machine can serve a request.
Use the pilot’s output, capacity, and recovery results to decide whether B3IQ fits your inference workload before moving production traffic.