Every AI founder eventually has to decide how much of the intelligence layer the company should actually own.
A startup can train a model from scratch, fine-tune an existing model, or build its product around an external API. All three approaches can produce impressive demos. But underneath the interface, they create very different businesses.
The decision affects capital requirements, engineering headcount, iteration speed, infrastructure complexity, gross margins and ultimately defensibility. For founders, the useful question is not simply which model performs best. It is which part of the AI stack creates enough strategic value to justify owning it.
Research into model scaling, fine-tuning, retrieval and inference systems suggests that startups should resist the instinct to build more infrastructure simply because they can.
Building a Model Means Committing Capital Before Product-Market Fit
Training a foundation model from scratch is fundamentally a resource-allocation decision.
A major NeurIPS study on compute-efficient model training examined more than 400 language models ranging from 70 million to more than 16 billion parameters, trained on between 5 billion and 500 billion tokens. The researchers concluded that compute-optimal training requires model size and training data to grow together rather than simply increasing the number of parameters.
Their resulting model, Chinchilla, used 70 billion parameters and four times more training data than the 280-billion-parameter Gopher while consuming the same training compute. Chinchilla then outperformed Gopher, GPT-3, Jurassic-1 and Megatron-Turing NLG across a broad range of evaluations.
The finding matters commercially because it demonstrates that competitive pretraining is not just about acquiring GPUs. Teams must make coordinated decisions about data quality, dataset size, architecture, compute budgets and training duration.
For a foundation-model company, owning that process may be the product.
For an application startup building AI for accounting, legal workflows, cybersecurity or healthcare administration, reproducing general language capability may be a poor use of scarce capital unless the company possesses data or technical expertise that existing models cannot replicate.
Fine-Tuning Changes the Economics of Model Ownership
Fine-tuning provides a middle ground: start with a capable pretrained model and adapt its behavior to a narrower problem.
The influential LoRA technique, published at ICLR 2022, works by freezing the pretrained model and inserting smaller trainable low-rank matrices rather than retraining every parameter. Microsoft Research notes that, compared with conventional GPT-3 175B fine-tuning using Adam, LoRA reduced trainable parameters by a factor of 10,000 and GPU-memory requirements by a factor of three.
The researchers also reported performance on par with or better than full fine-tuning across models including RoBERTa, DeBERTa, GPT-2 and GPT-3, while avoiding additional inference latency.
QLoRA pushed that cost curve further. The NeurIPS 2023 paper demonstrated that developers could fine-tune a 65-billion-parameter model on a single 48GB GPU while preserving full 16-bit fine-tuning task performance.
Its Guanaco model reached 99.3% of ChatGPT’s performance on the Vicuna benchmark after 24 hours of fine-tuning on one GPU, although the researchers themselves warned that existing chatbot benchmarks were not sufficiently reliable for measuring real-world chatbot quality.
That makes fine-tuning strategically interesting for startups whose advantage depends on repeatable model behavior: specialized terminology, predictable output formats, domain-specific classification or highly specific workflows.
The startup owns more of the behavior without paying to recreate the underlying intelligence.
Proprietary Knowledge May Belong Outside the Model
Founders also need to distinguish between changing what a model does and changing what it knows.
Those are different architecture problems.
The peer-reviewed NeurIPS paper that introduced Retrieval-Augmented Generation highlighted the difficulty purely parametric models have with updating their knowledge and providing provenance for their answers. RAG instead combines a pretrained language model with an external information-retrieval system.
In its experiments, the approach achieved state-of-the-art performance on three open-domain question-answering tasks and produced more specific, diverse and factual language than a parametric-only sequence-to-sequence baseline.
For a startup, this creates an important product distinction.
A legal AI company dealing with regulations that change every month may gain more from maintaining a carefully indexed and auditable source library than repeatedly fine-tuning a model to memorize every revision.
A cybersecurity startup may similarly want its system retrieving the latest vulnerability information rather than encoding a snapshot of that information inside model weights.
The defensible asset can therefore be the data pipeline, retrieval layer and workflow, rather than the foundation model.
APIs Let Founders Buy Speed Instead of Infrastructure
Using an external model API gives a startup less control, but it can purchase something extremely valuable during the early stages of a company: time.
The engineering burden hidden behind self-hosted inference is substantial.
An open-access paper published in the ACM Symposium on Operating Systems Principles found that large-model serving systems struggle because the key-value cache required for requests is large and changes dynamically, creating memory fragmentation and limiting the number of requests that GPUs can process simultaneously.
The researchers built PagedAttention and vLLM to address that problem and reported two to four times higher model-serving throughput at the same level of latency compared with systems including FasterTransformer and Orca.
UC Berkeley’s Sky Computing Lab similarly describes vLLM as achieving near-zero waste in KV-cache memory through more efficient memory management and sharing.
This is why “we will just host the model ourselves” is not a trivial engineering decision.
Self-hosting means operating GPU infrastructure, managing batching and memory, handling model upgrades, monitoring latency and reliability, and maintaining enough utilization to make the economics work.
An API lets an early-stage team postpone those problems while testing whether customers actually value the product.
The Cheapest Option at Launch May Not Stay Cheapest
The decision should not be permanent.
An AI startup might begin with an API because iteration speed matters more than infrastructure margins. As its proprietary dataset grows, it might add retrieval. Once customer workflows become predictable, fine-tuning could improve consistency. At sufficient inference volume, moving some workloads onto self-hosted models may become economically attractive.
Only later might training a proprietary model make sense—and only if model ownership has become strategically important enough to justify the capital.
That progression also avoids a common founder mistake: optimizing infrastructure before proving the product.
The relevant economics change with scale. API dependence that is negligible at 1,000 users can become material at 1 million. Conversely, infrastructure that appears cheaper per token can become expensive when GPUs sit idle or require a specialized engineering team to maintain.
The Real Moat Is Knowing Which Layer to Own
The strongest AI companies will not necessarily own their entire stack.
They may own proprietary data because competitors cannot reproduce it. They may fine-tune models because customers demand consistent behavior. They may control retrieval because freshness and provenance are critical. They may eventually operate inference because scale makes optimization valuable.
But owning a foundation model simply for the sake of independence does not automatically create defensibility.
A founder who spends millions building a model customers cannot distinguish from an API-powered competitor has created infrastructure—not necessarily a moat.
The more disciplined product strategy is to identify where the company is genuinely different.
For some startups, that difference will eventually live inside model weights. For many others, it will exist in data, workflow design, distribution, integrations, evaluation systems or deep knowledge of a particular industry.
The most useful rule for AI founders may therefore be simple:
Own the layer that makes the product difficult to reproduce. Rent everything else until the technical, economic or strategic case for ownership becomes impossible to ignore.
