Comment Re:Pfff, my 2009 iMac can run at 212F/100C (Score 2) 15
Yes! Finally someone who actually understands what this thing was actually designed for. I got one right out of the gate as I want to fine tune / distill expert models for eventual production use. My customers all run on CUDA, so CUDA is an absolute must for testing and development. Eventual deployment will be on much more realistic hardware hosted in the cloud. Developing in the cloud making lots of mistakes adds up fast dollar wise, and is honestly a bit clunky. Developing locally let's me make lots of mistakes. Heck, I kick it off and go do other things and check back later all for the one time sunk cost. I can also see that perhaps my reward model is not functioning properly and just abort it as it isn't going where I want to. A giant gaming PC would cost me more, and not fit in with what I am doing for the most part. I like a purpose specific machine for this work that has the full support of Nvidia and it's stack of software.
And to be clear, it absolutely can run quite a few models for experimentation, once again cheaply. I don't have to do anything but spin up a python virtual environment or use an existing UI based tool and play around. Sure, performance isn't great, but it let's me evaluate things on a very consistent and solid platform. I'm not going to run production inference on something like this for many reasons, just like I would not do that on a gamer machine.
For what it is worth I ran for 3 hours straight at 100% GPU utilization and saw no thermal throttling. It was putting out some decent heat for sure, but it was humming along. Also, I will note, I was doing this off grid at 10,200 feet on pure solar power in a smaller cabin. Try that with a gamer PC!
And to be clear, it absolutely can run quite a few models for experimentation, once again cheaply. I don't have to do anything but spin up a python virtual environment or use an existing UI based tool and play around. Sure, performance isn't great, but it let's me evaluate things on a very consistent and solid platform. I'm not going to run production inference on something like this for many reasons, just like I would not do that on a gamer machine.
For what it is worth I ran for 3 hours straight at 100% GPU utilization and saw no thermal throttling. It was putting out some decent heat for sure, but it was humming along. Also, I will note, I was doing this off grid at 10,200 feet on pure solar power in a smaller cabin. Try that with a gamer PC!