The What of my AI Computer
Most simply, it’s a computer built to run AI programs. It allows me to put AI models onto this computer, and run them within my own network, meaning I’m not sharing data out to AI companies. This also gave me some good experience putting together a computer that’s designed for running AI workloads, something I’ll need to understand for work. Why couldn’t I just use my existing computer? I could, but it wouldn’t be scalable, and I’d be fairly limited to what types of AI models I could run. My goal here was to start small with a simple workstation footprint, but be able to reuse as many components as possible if I want to expand to a server rack footprint. AI workloads require graphics cards (GPUs) to efficiently run the models, and in general you need a lot of memory and computing power on those GPUs. I have started with a single Nvidia RTX 6000 Ada with the goal to continue to add more along the way. This is the single most expensive component of the build, but also the most important. This will give me the capability to run some decent sized models efficiently on my machine. I can pack 3 of these cards into the machine until it’s time to upgrade, and when I am ready for a 4th card I will step up to a server rack case that will give me 8 GPUs worth of capacity.
What else is in this machine? To start it’s a Fractal Meshify 2XL case because I needed to have room for a server sized motherboard inside of it. The reason why is I went with an AMD Epyc 9354 32 core CPU that will pack a hefty computing punch. This CPU was picked so when I move to a rack chassis I can use a dual CPU board which will give me an even bigger compute punch. I’m cooling it with a 360mm AIO water cooler from SilverStone that will keep this big CPU cooled off. The CPU is paired with 256gb DDR5-4800 memory which should be plenty as I work up to 3 GPUs. All of this sits on a big AsRock motherboard that has 4 PCIe5 x16 lanes, 2 PCIe5 x8 lanes, 2 m.2 slots, 8 slots for RAM, and a couple 10GbE ports. And to power everything I have a chicken fryer, I mean a 1600w Seasonic platinum grade power supply.
The Why of my AI Computer
I began using Chat-Gpt pretty heavily when gpt-3 was released back in 2020, and continue to use Open AI’s latest model, GPT-4o a lot. GPT-4o is how I was able to build a variety of things on this website, letting it do the React coding for me. It also handles all my python coding with minimal knowledge required by me. Is the code perfect? Unlikely. Does it work for what I’m doing? Absolutely. One of the downsides of Open AI, and basically all private LLMs is that your data is considered public the moment you feed it into the chat bot. Open source models that you can download and run locally on your machine have lagged behind these private models, but that tide is turning a bit with Meta’s push with Llama 3.1. There are plenty of other models too, but Llama is one of the most popular. I want to be able to run models on my own machine, and be able to make fine tuning adjustments as I want to. I want to actually learn how these models work, and get to tinker with them. I’m a very curious learner, and I’m not afraid to break things. I want to push stuff until the slider comes off in both directions. This is why I’m starting my own little home lab off with an AI compute machine. I get to tinker with technology that will have huge impacts on our lives for the foreseeable future, and I get to learn it on my own terms.
Another big reason is that I’m a believer in understanding any product that I am involved with. I know a fraction of a percent about machine learning and AI. I work for an AI computer vision company, so I need to catch up and learn more of the basics. So far I’ve been loving every moment, getting to apply my business knowledge and process problem solving brain to projects at Plainsight. By tinkering on my own machine in my free time I’m gaining valuable knowledge that will help me at work in applying it to actual business problems. I’m gaining hardware knowledge that will be great knowledge as one of the core pillars for Plainsight is customers running our models on their own hardware. We believe in data privacy, and that customers should be able to run the product on their hardware whether that’s in the cloud or at the edge on their physical devices. As we work towards having hardware recommendations I’ll have gone through the process myself of configuring a machine, and understand at a better level what these customers will need. Now if only I can convince the IRS to let me write this whole thing off.
So What Now?
Learning Proxmox has been my current start to what now. My plan for the moment is to run Ubuntu Server 22.04lts for my AI VM. There will also be another smaller Ubuntu server running on the machine that takes care of some home services I can run locally. I will probably keep my little Intel NUC as my linux box that I use for VScode which is what I use to write everything for this site. This is a perfect job for that small but powerful box. But I’ll figure out more about what I’m doing with this machine, how I can break things, and what’s next as I get to tinkering. I’ll continue to post more updates, and I may even switch the blog posts over to headless wordpress. Whoops, spoiler alert! Read the website updates for some fun updates about the site, and how I’ve made it more than just a blog about my professional life.
Website Updates
I have added a recipes page that pulls it’s content from a wordpress site. My wife, Ally, and I have been cooking the majority of our meals at home since we moved in together back in 2016. Ally would do a weekly shopping trip, so we’d plan out our week’s worth of dinners every week going on 8 years now. This lived on her phone in a single notes page, which was starting to become difficult to navigate when we needed past inspiration. It also didn’t house recipes, just shopping lists and dinner ideas. Enter the recipes page, where we can go for dinner inspiration, the recipe, and it’s easy to share with others when they want to know how to make what we’ve made for dinner. I also wanted to get some experience working with headless wordpress and Astro, and this offered me the chance to do that. There are a few different guides on how to do this, and I was able to follow along with it. The only issue is that they don’t cover pagination, so I used Chat-GPT to help me build a React component for it. I also had Chat-GPT build a search function to make it easy to find a recipe. Each recipe comes with category tags that you can click to find like recipes. In the future I plan to add comments, ratings, and category filters to it. That’s all for now, thanks for reading!