I feel there is a catch here. I forgot mixture of experts was a thing -_-. They only activate like 32B at a time.
But this can be incredibly good, lets see!
Btw “and train” people missed the very very key word.
EDIT: @_@ training qwen 3.8 on under 6 GB. Yes, this seems VERY acceptable! I can see the endless potential here.
Nah, it seems it can finetune a 128 GB model with 6 GB RAM, I don’t see evidence that it can finetune something of 1.4T or more. I was excited for nothing.
Comments
Call me when something lets me train and run my own 1.4T model locally, in about 120 years.
https://github.com/lyogavin/airllm
Those 120 years went by in a flash.
He is the world’s dumbest man afterall
I feel there is a catch here. I forgot mixture of experts was a thing -_-. They only activate like 32B at a time.
But this can be incredibly good, lets see!
Btw “and train” people missed the very very key word.
EDIT: @_@ training qwen 3.8 on under 6 GB. Yes, this seems VERY acceptable! I can see the endless potential here.
Nah, it seems it can finetune a 128 GB model with 6 GB RAM, I don’t see evidence that it can finetune something of 1.4T or more. I was excited for nothing.
Maybe just another two years, but the trajectory is promising and hardware will improve, as well. Strata is good for inference, at least.
Sign in to comment. Sign in