Comment on Apple has new Mac app coming to make running local AI models easy
You are viewing one comment and its replies. View all comments · View the comment above
You are viewing one comment and its replies. View all comments · View the comment above
Comments
https://github.com/lyogavin/airllm
Those 120 years went by in a flash.
He is the world’s dumbest man afterall
I feel there is a catch here. I forgot mixture of experts was a thing -_-. They only activate like 32B at a time.
But this can be incredibly good, lets see!
Btw “and train” people missed the very very key word.
EDIT: @_@ training qwen 3.8 on under 6 GB. Yes, this seems VERY acceptable! I can see the endless potential here.
Nah, it seems it can finetune a 128 GB model with 6 GB RAM, I don’t see evidence that it can finetune something of 1.4T or more. I was excited for nothing.
Maybe just another two years, but the trajectory is promising and hardware will improve, as well. Strata is good for inference, at least.
Sign in to comment. Sign in