fmstrat@lemmy.nowsci.comtoPrivacy@lemmy.ml•Running local LLMs for privacy as an alternative to ChatGPT, MS Copilot etc.?English
3·
2 days agoTo add to this, I run the same setup, but add Continue to VSCode. It makes an interface similar to Cursor that uses the Ollama instance.
One thing to be careful of, the Ollama port has no authentication (ridiculous, but it is what it is).
You’ll need either a card with 12-16GB VRAM for the recommended models for code generation and auto complete, or you may he able to get away with an 8GB card if it’s a second card in the system. You can also run on CPU, but it’s very slow that way.
Ollama works with AMD.