Pro to Technology@lemmy.worldEnglish • 2 months agoGoogle quietly released an app that lets you download and run AI models locallygithub.comexternal-linkmessage-square48fedilinkarrow-up1247arrow-down125
arrow-up1222arrow-down1external-linkGoogle quietly released an app that lets you download and run AI models locallygithub.comPro to Technology@lemmy.worldEnglish • 2 months agomessage-square48fedilink
minus-squareAmbiguousPropslinkfedilinkEnglish23•2 months agoThat’s fair, but I think I’d rather self host an Ollama server and connect to it with an Android client in that case. Much better performance.
minus-squareGreg ClarkelinkfedilinkEnglish4•2 months agoYes, that’s my setup. But this will be useful for cases where internet connection is not reliable
minus-square@OhVenus_Baby@lemmy.mllinkfedilinkEnglish2•2 months agoHow is Ollama compared to GPT models? I used the paid tier for work and I’m curious how this stacks up.
minus-squareAmbiguousPropslinkfedilinkEnglish1•2 months agoIt’s decent, with the deepseek model anyway. It’s not as fast and has a lower parameter count though. You might just need to try it and see if it fits your needs or not.
minus-squareGreg ClarkelinkfedilinkEnglish3•2 months agoHas this actually been done? If so, I assume it would only be able to use the CPU
minus-square@Euphoma@lemmy.mllinkfedilinkEnglish7•2 months agoYeah I have it in termux. Ollama is in the package repos for termux. The speed it generates does feel like cpu speed but idk
minus-squareDiplomjodlerlinkfedilinkEnglish3•2 months agoIs there any useful model you can run on a phone?
minus-square@gens@programming.devlinkfedilinkEnglish2•2 months agoLlama.cpp (on which ollama runs on) can. And many chat programs for phones can use it.
Why would I use this over Ollama?
Ollama can’t run on Android
That’s fair, but I think I’d rather self host an Ollama server and connect to it with an Android client in that case. Much better performance.
Yes, that’s my setup. But this will be useful for cases where internet connection is not reliable
How is Ollama compared to GPT models? I used the paid tier for work and I’m curious how this stacks up.
It’s decent, with the deepseek model anyway. It’s not as fast and has a lower parameter count though. You might just need to try it and see if it fits your needs or not.
You can use it in termux
Has this actually been done? If so, I assume it would only be able to use the CPU
Yeah I have it in termux. Ollama is in the package repos for termux. The speed it generates does feel like cpu speed but idk
Is there any useful model you can run on a phone?
Try PocketPal instead
Llama.cpp (on which ollama runs on) can. And many chat programs for phones can use it.