Running AI Models Locally in Your Browser
Local Mode lets you run open-source AI models entirely inside your browser tab. Your prompts and the model’s replies never leave your device - no server sees your conversation.
Prerequisites
- A modern desktop browser (Chrome, Edge, Firefox, Brave or Safari)
- Enough free memory (RAM) for your chosen model - each model shows its estimate
- A connection fast enough to download the model once (a few hundred MB to about 2GB)
How It Works
Local Mode runs llama.cpp, compiled to WebAssembly, inside your browser tab. It loads models in the GGUF format, downloaded directly from Hugging Face. Inference currently runs on your CPU, so a dedicated GPU isn’t required.
Benefits:
- Complete privacy - prompts and replies are processed on your device
- No API costs or rate limits
- No account required
- Keeps working without an internet connection once a model is loaded
- Each model downloads once and is kept in your browser
Limitations:
- Slower than cloud models: the smallest models write a few words per second, larger ones about one
- Limited to small models (up to about 3B parameters), which are less capable than frontier cloud models
- Text only (no image input)
Getting Started
Step 1: Open Local Mode Setup
Click Local Mode on the start screen, or open the sidebar and click Local Mode in the Tool Belt section.
Step 2: Wait for the Engine
The setup window loads the local engine and confirms that models run on your CPU, inside the tab. No GPU is needed.
Step 3: Choose a Model
Pick a model from the dropdown. Models are grouped into categories, and each one shows its estimated memory requirement in parentheses.
First time? Start small. A model under 1 billion parameters downloads quickly and responds fastest. Larger models give better answers but need more memory and run slower.
| Model size | Approximate memory | Good for |
|---|---|---|
| Under 1B | Under 1GB | Quick answers, testing |
| 1-3B | 1-3GB | Everyday questions, drafting, summaries |
The in-browser engine can use at most 4GB of memory, so models much larger than about 3 billion parameters won’t load, even on a powerful machine.
Step 4: Download the Model
Click Download & Enable. The progress bar shows:
- Downloading the model file from Hugging Face (with MB downloaded so far)
- Initializing the inference engine
- Loading the model into memory
Keep the tab open until it says Model ready!
Step 5: Start Chatting
Once a model is enabled, a Cloud/Local toggle appears next to the Model header in the sidebar. Click Local, pick your model, and start typing.
Local Mode remembers which models you enabled and whether you were in Local mode. The model file stays in your browser’s storage, so after a reload, selecting it loads it from your device in seconds instead of downloading it again.
Switching Between Local and Cloud
After enabling at least one local model, the sidebar shows a toggle:
- Cloud (default) - Uses cloud models through Obscurify’s anonymizing proxy
- Local - Uses your enabled models, running in your browser
When in Local mode:
- Your selected local model is shown in the input placeholder
- Responses are labeled with the model name + “(Local)”
- All processing happens on your device
Troubleshooting
Download fails or stalls
- Check your internet connection and try again
- Some networks block Hugging Face downloads; try another network, a VPN or Tor
The tab crashes or the model fails to load
- The model needs more memory than is available - choose a smaller model
- Models much larger than 3B parameters exceed the browser engine’s 4GB limit
- Close other memory-hungry tabs and apps
Responses are slow
- Choose a smaller model
- Keep the tab in the foreground while it generates
Poor response quality
- Smaller models have real limits - try a larger one (up to about 3B parameters)
- Switch to a cloud model for harder tasks
- Local models do best with straightforward questions, summaries and short code snippets
Privacy Note
When using Local Mode:
- Model files are downloaded directly from Hugging Face, which sees your IP address and which model you downloaded - but never your prompts
- Downloaded model files are stored in your browser’s storage on this device
- All inference runs locally in your browser
- No conversation data is sent to any server
- Your conversation history is saved in your browser’s local storage, not on our servers
To remove a model and delete its file, click × next to it in Local Mode Setup. To clear everything, clear your browser’s site data for Obscurify.ai.