Looking for a recommendation on an LLM that can guide a user on how to manually set up a home server that runs Jellyfin, Arr Stack, etc. Have a couple of friends interested and I have some spare old NUCs I was going to give them, so I was going to download the software for them and then walk them through it.
But it’ll be difficult to align our schedules for the next month or so (school starting and different time zones) so then I thought maybe I could just run an LLM they could use on my main PC for when they have time.
Why not just use one of the existing services? Those cost extra money and none of us want to financially support those companies.
You must log in or register to comment.
Either way OP, you will need to install something like Rust, because you will become the network admin/IT dept.
Already planning on them adding Rustdesk in case of issues in the future actually 😅
Interesting question. If you are asking for an LLM (that is self-hosted and can do that?), you’re going to need to provide some significant tooling, like rag / documentation, troubleshooting, sort out concurrency, front end etc. Honestly…it just easier to point them at a YouTube (network chuck has good stuff).
It absolutely can be done and it absolutely can be valuable - for you personally. But if they’re having trouble doing basic things like installing jelly fin, they have zero chance of doing something like that themselves.
Honestly, I think your easiest option for your non-technical friends is just to point them at one of the cloud providers, like chatGPT or Claude.
OTOH, how much work are you willing to put into this and what’s your GPU / LLM set up like? There
Your basic foot in door starting point is going to be installing and provisioning OpenWebui, getting a good local model up and running (Qwen3.6-35B or Qwen3.6-27B) and creating a “Knowledge Base” in OWUI with requisite documentation. You’ll need to set up tailscale / headscale so they can access your OWUI instance from their homes, too.
If you’re serious about this, write back and I’ll thumbnail sketch it out for you. It’s a good project and I’ve done similar. There are real complexities to something like this beyond just “install ollama, lol done”.
I am interested in this in general too. I have a 3090 which I know isn’t much but have been thinking of trying to get a second 3090 and using an NVLink to increase to 48Gb. Mostly for learning and testing purposes, and then to eventually offer to donate to my wife’s local school district a local model they own (since teachers aren’t supposed to put student info online, that includes private LLM models like ChatGPT, but they keep accidentally doing so anyway).
But one project at a time, and setting up this LLM would help my two friends if they have issues (it’s easy to forget for people like us that to nearly everyone else, setting up Jellyfin properly with remote access as well as things like Docker are not easy, even when they have base level knowledge), but it might help others in the future too that might then get interested in self hosting. A 24/7 guide can make a difference I think in someone actually trying self hosting or not.
I’ll give a non-answer; if they dont spend the time setting it up themselves to understand it, they are kinda screwed.
There are TONS of videos out there, step-by-step guides, that should be enough honestly. I understand the appeal of LLM though to talk through stuff, but honestly, just give them a good guide. They can ask Googles AI if they need terms explained etc.
Just a different perspective.
I’ll give a non-answer; if they dont spend the time setting it up themselves to understand it, they are kinda screwed.
I’d give that a +1. Anytime you put a little sweat into something, it makes it a bit more valuable to you.
Any step-by-step text guides you’d recommend? They’re not exactly tech beginners, but they’re also not quite knowledgeable yet. I guided them through Linux install but they use Bazzite and mostly stick to Bazaar on it for any programs they need.
Maybe a easier place to start is a beginner friendly system? Lots of people like running Unraid for example, and they have lots of apps that will make it easier to get into (https://ca.unraid.net/), that might help ease them into it. Being thrown into a Linux terminal might feel daunting, but also a great way to learn.
I think many just need to get hooked on it, then they more naturally feel like they want to customize and try out stuff.
I dont have any guide recommendations, since its been a while since I started. You can probably do some searching on the stuff you want them to setup.
I asked mostly because I haven’t seen any guides like that. The long video guides are also usually outdated or a very specific method that’s more advanced (like bare metal setup or using ignition)
My plan was to lead them through something like what I use which is base Debian using Dockge and a reverse proxy (though I’m looking into KDE Plasma BigScreen). I think they might be able to figure it out with help and some general written instructions and the AI for further questions.
For the arr stack the best resource I know is https://wiki.servarr.com/ which I referenced for my setup in Kubernetes.
Most projects have pretty good documentation on how to setup their services, in many cases its just the Docker config afterall.
It’s a good reference guide, yes, but it’s not good for beginners. The linked TRaSH guide seems better but I think are still a lot.
The issue I’ve found when searching for resources for them is that there’s really no guide me for people who know very little but want to do this and guides them in the proper steps from beginning to end. I have an IT degree and even I remember struggling a little bit when I first delved into self hosting because I never really studied networking, and networking feels like a different field entirely. Right now self hosting in general reminds me of the early days of Slackware (maybe even SLS) Linux (for those that remember) - you kind of have to ask around and figure things out on your own through a bunch of lookup.
I’m trying to ease the friction a little bit for those that want too start, and want to see if an LLM that can quickly define some words and tell you not only where to start but the steps after that based on your skill/knowledge level can ease some of that friction.
At least until one day self hosting becomes about as easy as modern Linux.
Is it just for local usage on their network or will their be remote access via the internet? What OS are they installing on? Are they familiar with installing the OS?
In my experience Jellyfin is an easy install and easy to manage.
I run a few servers on an old laptop and I’ve messed up some random other shit on the laptop then wiped it one time, moved to another OS one time, moved to another machine another time. So I’ve setup Jellyfin I think 4 times now.
Overall it’s pretty easy in my opinion.
- Find your ip- ifconfig
2.Log into router and reserve that IP so it stays with the device. (This will vary by internet service provider, if unsure look at the gateway you found in the last step, if it says 192.168.1.254 or such, that’s it)
- If using it local punch a firewall hole for 8096 so you can get to it from say your Android/Roku TV, LG TV, phone, computer, whatever.
(Standardly something like: sudo ufw allow 8096)
- Terminal command listed here for the install: https://jellyfin.org/docs/general/installation/linux/
If you like using docker like many do then you may prefer to do it in a container.
Anyways that should bring it up. Then go to http://the.ip.address.found:8096/ and go into the dashboard to set up users, pick what access they have.
Also where you’ll set up the libraries, so say “Movies” and point it to where you store them… you can point it to multiple directories
Oh and if a laptop change the close lid settings to keep machine on when plugged in and lid closed. Set all power settings to not go to sleep.
On an aside I set my VPN up as a split tunnel, easy, just Google your VPN and split tunnel, should only take a few clicks. I set qbittorrent and firefox traffic to go through the VPN, then I have everything else stay on local network.
I install Rustdesk on my phone and laptop, then remotely access the device wherever I’m at. Use the search function in qbittorrent, tell it to download sequentially, and download to the movies directory I want, launch the jellyfin app on my phone, hold my finger down on the movies folder, tell it to refresh meta-data and then launch the jellyfin fin app on one of the tvs.
Some time from “do you want to watch ____” to starting to watch it is ~3 minutes. If the movie downloads faster than you watch it, should have no hiccups
why would anybody think that setting up a local “LLM that can guide a user on how to manually set up a home server that runs Jellyfin” would be easier than setting up jellyfin itself?
🤔
just follow the instructions on https://jellyfin.org/docs/general/installation/linux/#debuntu-debian-ubuntu-and-derivatives-using-apt
And the Arr Stack, and KitchenOwl, and LibreCloset. They also want something for YouTube, but haven’t decided what they might like the most yet - was thinking of testing KDE’s new Plasma BigScreen and seeing how Waydroid does at running S-tube.
If that worked well enough, then I’d probably try setting them up with that
Please update your title with [AIT] as noted in the sidebar.
GPT4All can accommodate a plethora of models all the way until you run out of VRAM. LOL Maybe play around with some of them. Right now I’m experimenting with DeepSeek R1 Distill Qwen 14 B, and Qwen 2.5 Coder -32B Instruct GGUF. Pretty neat stuff.
Is GPT4all still up to date though? I see the last release was in 2024
Release v3.10.0 dropped on Feb 24, 2025, so yeah it does have a little age. I’m not sure how often they would need to update the front end, as it seems the models are the driving force.
Mature software doesn’t need hourly updates. All the shit that leverages the library repos in horribly unsafe ways - ohai npm - that stuff needs constant updates.
I know it doesn’t need hourly updates but something that relies on other data (in this case LLMs) probably needs yearly updates at the very least to ensure better compatibility with newer models.
Someone started a stack named admirarr but it miss the configuration of many parts. If you find something keep us updated.
Easy setup would be to use koboldcpp + SillyTavern + Gemma 4 26B A4B GGUF of the largest quant your graphics card can fit together with your context.
Remember to setup SillyTavern to allow network connections and create a user and password, default installation is localhost only.
Set the temperature to 0.3 if using IQ3_M, higher quants allow higher temperature, and make sure all the formatting templates inside Advanced Formatting are set to Gemma 4.Then use either Gemma 4 itself or the free google.com AI to create some “W++ Denze with Horizontal Lines summaries for Gemma 4” of the relevant documentation of the latest version of the home server apps your friends are gonna install.
You should ensure the resulting lorebook entries are no more than 1k tokens each to leave some context for your friends chats, add it to a character card in SillyTavern and set some keywords to allow them to load dynamically and not stay in context memory all the time.Regarding the character card you can ask Gemma 4 to write that for you too, I find “Write a character card in W++ Denze with Horizontal Lines style for Gemma 4 with this name, personality, attitude and skillset” works well for that.
Then ask it to write a “First message prompt that starts with X, continues with Y and ends with Z for that character card” and you get a first draft to rewrite and paste into the “First Message” of the character card. The first message works as a template that Gemma 4 will imitate when you chat with it. Then simply try the chatbot out before letting others use it.I find my own Gemma 4 26B A4B IQ3_M works well for practicing hiragana and katakana, discussing programming or troubleshoot existing code or writing a small function but it can’t be expected to write a correct DatabaseService.cs from scratch and stuff like that.
Oh and forget about finding good cards and lorebooks for SillyTavern use online, most users use it for NSFW Roleplaying chats. It is a very easy UI to use to create harnesses for your local LLM though.
Depending on the use and the size of your graphics card you might be better of trying Gemma 4 12B too.
On my Nvidia 5060 Ti 16GB I can use either Gemma 4 A4B IQ3_M (larger model but highly compressed) or Gemma 4 12B Q8_0 (smaller model but much less compression) and after a night of testing with the Gemma 4 12B Q8_0 I’d say it’s a better helper when it comes to coding but a worse gm for my dnd inspired fantasy world roleplaying adventure.I imagine your “setup a selfhosted environment” is closer to the first.
Probably setup ollama with qwen and aider first. Then give it the docs for the things you want to set up.
I have the Alpaca Flatpak, can I set this up in that or is Ollama better? I’m running on Debian
Never used alpaca. So I can’t help there.
