There are about ten things going on at once in my data universe right now.
-
I’ve been building up to the project of switch from Mac OS back to Linux as Mac OS and Windows seem to both be dying, in different ways. I’ve been running big Hackintosh boxes since 2009, and I’m currently on Ventura. One step from the end of the line in a couple of years. The switch isn’t done yet; there’s a ton of data and organization to sort through, and apps to replace. But it’s going to have to be done.
-
I’ve been building up an OpenClaw to play with. The weird thing is that it may be obsolete already. I’m not quite sure how you’re meant to build workflows that you use for more than a week at a time right now, as AI is changing so quickly that basically it’s rip-and-replace your entire universe every Friday. But I’m going to try to stick with this one or something similar for a moment, until I figure out how to migrate agents between platforms and what that might even mean.
-
I’ve also been working toward hosting models locally, because burning tokens at a decent clip is expensive. To that end, tonight I replaced my trusty RX480 with an RX6700XT (12GB VRAM without breaking the bank) and got it up in Linux and in Mac OS. It went deceptively smoothly.
-
Likely this weekend, I’ll rehome the entire setup into a new case, one with enough length to hold the V620 card I’m about to put in it—32GB VRAM. The cooler shroud is already printed in ABS and the cooler is mounted up. We’re ready to go, we just need to rebuild around a 1200 watt power supply and a larger case that can actually hold it. At that point, we’ll have 44GB VRAM and 128GB DRAM to play with.
How do I know I’m probably too old for this? Because I think I feel more excited about seeing the graphics display on a new card than I do about filling its VRAM with LLMage. But oh well. The goal is to eventually migrate back to Linux full time (and from Lightroom to Darktable, and from Devonthink to a custom vector search space with an LLM interface), and then to run a decently sized local model with some sort of mix of experts configuration for the next few months. If all goes really, really well, maybe we pick up another V620, in which case w’ll have 76GB VRAM.
All of this is a way not to be left behind due simply to not wanting to afford to pay through the nose for tokens in the short term. Of course I’ll still have to pay in other ways (energy, hardware, etc.) but it should be less for full-tilt operation, and running it locally opens up some interesting experimentation possibilities.