A closed API is faster to ship. An open model is what I want when data cannot leave the building, or when per-token pricing will not work at the volume I expect.
“Open” still means read the license. Some weights are free to try and not free to ship in a product.
Open vs closed
| Closed API | Open weights | |
|---|---|---|
| Start | An API key | A download and a runtime |
| Data | Leaves your server unless the contract says otherwise | Stays where you run it |
| Quality | Often the best general chat | Varies. Test your task |
| Ops | Their uptime, their price changes | Your GPUs, your patches |
Popular open source models change every few months. I do not memorize the leaderboard. I keep a short list for chat, embeddings, and speech, and I re-benchmark when I am about to commit.
Finding them
Hugging Face is the index I use.
- Hugging Face Hub — model cards, files, licenses, and how people actually run the model
- Hugging Face Tasks — filter by what you need: text generation, feature extraction, automatic speech recognition, text-to-image
- Using open source models — read the card before you celebrate. “Open” and “commercially usable” are not the same sentence
How I run them
Pick the smallest runtime that matches where the feature lives:
- Inference SDK / hosted inference — someone else runs the weights, you still get an API. Less ops, less privacy than local.
- Ollama — local chat models for development and for products that must stay on device or on your VPC. Ollama models are a curated set. The Ollama SDK is enough for a first integration.
- Transformers.js — when the model should run in the browser or in Node without Python. Good for small models. A bad idea for a 70B chat model.
A rule I use
If the feature is a prototype and the data is not sensitive, I use a closed API. If a lawyer or a customer contract says prompts stay in our network, I move that path to an open model and I accept the quality trade until evals say otherwise.
Next
Search by meaning starts with vectors, not with a bigger chat model: