I keep looking at mac mini’s and wondering if I should run models locally. But they’re always so expensive I don’t really need to run AI models at all it’s just an idle curiosity.
I think I’m going to ignore AI until it either gets cheaper or builds us a warp drive.
I feel there is a catch here. I forgot mixture of experts was a thing -_-. They only activate like 32B at a time.
But this can be incredibly good, lets see!
Btw “and train” people missed the very very key word.
EDIT: @_@ training qwen 3.8 on under 6 GB. Yes, this seems VERY acceptable! I can see the endless potential here.
Nah, it seems it can finetune a 128 GB model with 6 GB RAM, I don’t see evidence that it can finetune something of 1.4T or more. I was excited for nothing.
Thyey aren’t 3 years behind on custom hardware to run local AI models.
E.g nvidia’s DGX Spark came out in late 2025.
Their MacBook pros have been running models for years just fine, but this is more purpose built.
Edit: Also the Studio Ultra’s have been running AI models for awhile as well with their higher memory bandwidth and total memory allowed, but they’re very expensive.
Comments
Even easier than using LMStudio? I wonder how they want to manage that.
I keep looking at mac mini’s and wondering if I should run models locally. But they’re always so expensive I don’t really need to run AI models at all it’s just an idle curiosity.
I think I’m going to ignore AI until it either gets cheaper or builds us a warp drive.
Them being a great deal is gone
Call me when something lets me train and run my own 1.4T model locally, in about 120 years.
https://github.com/lyogavin/airllm
Those 120 years went by in a flash.
He is the world’s dumbest man afterall
I feel there is a catch here. I forgot mixture of experts was a thing -_-. They only activate like 32B at a time.
But this can be incredibly good, lets see!
Btw “and train” people missed the very very key word.
EDIT: @_@ training qwen 3.8 on under 6 GB. Yes, this seems VERY acceptable! I can see the endless potential here.
Nah, it seems it can finetune a 128 GB model with 6 GB RAM, I don’t see evidence that it can finetune something of 1.4T or more. I was excited for nothing.
Maybe just another two years, but the trajectory is promising and hardware will improve, as well. Strata is good for inference, at least.
Only 3 years behind. Getting better, Apple
Thyey aren’t 3 years behind on custom hardware to run local AI models.
E.g nvidia’s DGX Spark came out in late 2025.
Their MacBook pros have been running models for years just fine, but this is more purpose built.
Edit: Also the Studio Ultra’s have been running AI models for awhile as well with their higher memory bandwidth and total memory allowed, but they’re very expensive.
Now there’s a name I haven’t heard in a long time.
FileMaker is still going strong
I give it a week before someone reverse engineers it and uploads a foss option.
Idk if I hate this kind of corpo project if they’re paying people for their work over more FOSS-slop
There are already a lot of FOSS options that run on Apple Silicon.
it is called install asahi or buy a proper computer
Sign in to comment. Sign in