I’ve been a Software Engineering Student for 2 years now. I understand networks and whatnot at a theoretical level to some degree.
I’ve developed applications and hosted them through docker on Google Cloud for school projects.
I’ve tinkered with my router, port forwarded video game servers and hosted Discord bots for a few years (familiar with Websockets and IP/NAT/WAN and whatnot)
Yet I’ve been trying to improve my setup now that my old laptop has become my homelab and everything I try to do is so daunting.
Reverse proxy, VPN, Cloudfare bullshit, and so many more things get thrown around so much in this sub and other resources, yet I can barely find info on HOW to set up this things. Most blogs and articles I find are about what they are which I already know. And the few that actually explain how to set it up are just throwing so many more concepts at me that I can’t keep up.
Why is self-hosting so daunting? I feel like even though I understand how many of these things work I can’t get anything actually running!
Each one of these things is a separate topic/project. There is of course, overlap, but concentrate on just one at a time. If you try to do everything, you’ll go mad. Application Administrator, Network Engineer, Sysadmin, Network Security, are often separate full time jobs. Just concentrate on your own needs, one at a time, then focus on how simple you can pare down that one thing you need/want to self host. What is the minimum set of capabilities you must have for this one thing? Then, expect to have as much research for this one thing as taking a 1 credit college course, at least. Maybe more, maybe less, depending what it is. That’s a more realistic set of expectations.
I’ve done a lot of web design and web hosting, and messing with Linux server stacks for around 30 years, now. You absorb a LOT over time, but it is all always changing. When I set up my first server, Cloudflare didn’t exist yet. VPNs weren’t a thing yet. The only people I knew with a network in their home were Computer Science professors. Wifi was not a thing yet. It isn’t you being inadequate, it’s you trying to do multiple jobs that each require constant professional development.
You don’t have to do it, 99.9999%^([citation needed]) of the population don’t. If it doesn’t interest or excite you then it’s not worth it.
That’s not bad
For me and I’m not sure if it’s been mentioned here already or not, but I don’t really have any programming skills or super geek out on this stuff. But I like the tools, right? So teaching myself some of this is hard, especially when there is an attitude with most of this stuff that you should know everything about docker containers and weird networking rules, etc. Sometimes people are helpful, more often than not, it’s a super short answer you have to interpret and decipher to figure out.
I self host for the same reason I play Factorio, it’s a constant stream of solving problems and adding new ways to do things. There is a lot to do, but you don’t need to do everything right away. Like a lot of people said, just break it up. I use Vikunja to create a list of things I want to and then break those things down into hour or weekend projects.
There is a challenge in the “bootstrap” process. But once you get something going then its easier to keep adding. Stuff like “Reverse proxy, VPN, Cloudfare bullshit” is not super necessary at first. Start with just getting an app going at an ip address and port that’s only accessible on you LAN, then add the other stuff.
If you don’t want to deal with these things and it’s just you or maybe a few others, here’s the easy way:
-
Define the services for each host (i.e. physical + virtual machine) in a single Docker Compose file. You can even put all of these docker-compose files in a single Git repo
-
Install Tailscale on all your machines - this is really easy and will let you securely access everything without having to forward ports or worry about the attack surface of the public web
-
Set up something like Heimdall or https://github.com/tailscale/golink to have easy to use shortcuts for your services
This gets you out of SSO signin, certificates, reverse proxy setups, all the things that are (understandably!) complex and annoying to set up
-
For VPN home labbing you should check out wireguard, it’s stupid simple and very powerful. Reverse proxy stuff I handle through nginx, mostly because I’m using it for web hosting anyways and I’m comfortable with the workflow. I don’t bother with cloudflare much because I host a jellyfin instance and I believe that’s against their TOS, but just take it one step at a time and you’ll figure it out in no time.
Also I can’t stress enough the ease of use of docker compose files for managing and quickly spinning up new apps.
Part of working with tech is knowing what to search for und using the right keywords. If you could give an example what guide / information you are unable to find, someone could give you an example how to search for it. I personally know a few junior devs and junior devops which use llama2 / chatgpt since they dont know how to search for it or read the docs.
I actually hate GPT, dislike it’s answers and find myself knowing better than it most times.
I’ve been trying to setup a DNS server to create my own domains internally within my VPN but I keep finding info on how DNS servers work, and how to make a records on registrars, but nothing on what I actually need to install and run to have my own DNS for example. Same thing goes for many other services, but that’s the one bugging me for the longest time because it should be so simple.
I’ve found plenty of tutorials on how to make a cache DNS, just not an authoritative name server btw, and I’ve searched for both DNS and name server to no avail. If it was Linux I’d write some custom rules in my hostfiles and be done with it, but it’s so much harder to do on Windows and that’s my daily use OS for now…
“self hosted DNS server” is what I searched for, but I also knew what I was looking for “Unbound” with PiHole is what I use.
Also, Windows has a host file that it uses. %windir%\System32\Drivers\etc\hosts
%windir% is the environment variable for your windows install directory. Usually C:\Windows
You can add your entries in there, just like Linux, though I am sure the syntax might be slightly different. I really haven’t played with host files on Linux, yet.
I hate the fact you gave your honest opinion and othets have to downvote you. I myself rarely use ChatGPT to find a direct answer, instead I use it to give me resources to find the answers for the questions I didn’t think about asking or knew existed. More like thinking outside the box or brainstorming. And always do your due diligence to verify any answer it gives.
@mooglestilzkin has already posted the url for Christian Lempa‘s YT channel. CL has just started a series „made for you“ about Docker: https://youtu.be/Nm1tfmZDqo8?si=g_L8bmm9qcFx9wve
Networking is extremely dense and extremely hard.
Its also a space filled with professionals who do it for a job. Many on this sub are current or ex networking or security professionals who understand this shit like the back of their hand.
Im no idiot with IT - I’ve had pc’s my whole life and work in close proxiimity to IT. But networking, security and the clusterfuck of everything associated with it is honestly about as hard to get into as actuarial science.
It requires such a heavy base of fundamental concepts that if you dont start at the bottom and work up, you have no clue wtf is going on.
I can’t get anything actually running!
Most of my time in my sysadmin work is sitting there staring at the screen reading error logs about why some new product isn’t working.
make sure you ACTUALLY have a solid grasp of networking. dig into the osi model or the 3 layer dod model. Go further than “yup. encapsulation. got it”. Look at the actual headers in layer 2 and layer 3. Figure out how packets get routed and how frames get forwarded. Look at how NAT actually mangles packets.
Understanding hos the traffic flows will make troubleshooting much easier.
Well, I can somehow understand you, but I didn’t have so many problems until now. Especially reverse proxy is in my opinion pretty well documented. I got everything running what I wanted, except LDAP. I can’t seem to understand this shit and there aren’t any good guides either :D.
If you’ve never gone and built your own system, or started a network from scratch without a container based system, and you’ve relied on stacking your own environment for a long time, you might necognize it as you would any skill.
I’m sure I could build a shed, for E.g. I know to get the wood, the panels, the cement, and all the things I’ll have all documented in front of me.
Then I will start, realize I forgot I also needed a permit, which I then found out I might also need to run power to it, so now shed has become something that doesn’t feel impossible, it feels daunting
Why? Well, I don’t build sheds, I don’t keep building things that would allow me to have all the framework, project of getting my shed built. So, as with a homelab, take it one step at a time, and if rhe shed isn’t the house, you’re allowed to skip a few things and stick with what works for you.
I use homelab and self hosting as a passion, but I also have 25 years of networking and systems administration experience so docker and containers just made my normal, tedious job of building up complex network infrastructure, feel like a job, and my homelab and self host happens to be a hobby
Treat your homelab as a passion or simple hobby, or use it as a skill building tool for work
Overall it’s just time, experience and expectations that either drive you forward or set you back
Back in the day you had to find a book if you didn’t know people
Decide what you want to self-host. Keep it simple at first then work your way up as need be. You could first self-host a simple web site. You could use Tailscale free vpn with 100.x.x.x ip’s to test your setup from another computer anywhere.