Comment on Discord is killing its Electron app that can eat 4GB of RAM on Windows 11, and the Rust rebuild uses just 290MB
You are viewing one comment and its replies. View all comments · View the comment above
You are viewing one comment and its replies. View all comments · View the comment above
Comments
Perhaps they run a local model, then tokens are essentially free.
Would need a very beefy cluster, and that definitely isn’t free.
Well yes, I’m not saying it’s a free lunch. The point is a large up-front investment buys you unlimited tokens in perpetuity for no more than the cost of electricity to power the cluster — which can be surprisingly cheap for UMA machines.
I’ve swapped off API models entirely and have been able to do a lot more useful work since I don’t have to worry about token cost or usage limits. Granted it’s slower per task (less tk/s), but overall much more productive, and private.
Sign in to comment. Sign in