UPDATE: To everyone who suggested YUNO, thank you so much. This seems like it is about to make my journey much easier. It is basically almost exactly what I was looking for, but I was unaware that it existed.
Thank you ALL for your suggestions, actually. It’s a bit overwhelming for an almost complete noobie but I an going to look into all of the suggestions in time. I just saw that there were several mentions of YUNO so I decided to make that one of the first things I investigated.
So, about two months ago, I had a very eye opening experience. As the result of a single misconfigured security setting on my Android, I was locked out of my Google Account on my phone AND all of my PCs. I had no access whatsoever to Google, or any of the literally hundreds of services that I get through Google.
This is when I realized that I relied entirely on Google/Android because those two days were actually very difficult, being cut off from media, services, passwords, everything, from the past almost twenty years of my life, could be taken away from me in an instant. The decades of my life that were locked away in my Google Account included hundreds of thousands of pictures, almost a hundred thousand audio tracks, several hundred books, several hundred apps, thousands of videos, etc. ad infinitum. Unfortunately, very little of this material was backed up at that point. That is my fault. Also, the misconfigured security setting was my fault as well.
The amount of data, media, memories, services, etc. that would have been lost is actually endless and it would have affected my life in several ridiculously negative ways.
Luckily, in the end, I was able to get my access back and then basically immediately grabbed all of the several terabytes of information and media of mine that they had, and that I was almost locked out of. I have it all in my house now on a drive in my computer, with a backup made on another disconnected disk.
I then decided that no corporation was ever going to have such an insanely high level of influence on and control over my entire life and my media ever again. That experience was actually very scary.
I’ve been trying to get into SelfHosting, but am finding it quite daunting and difficult.
There is a LOT of stuff that I have to learn, and I am mostly unsure of where to even begin. I know basically nothing about networking.
I need to learn the very basic stuff and work my way up from there, but everything that I’ve seen on the Internet assumes that the reader already has a basic to intermediate understanding of networking and the subjects that surround it. I do not, but I am going to learn.
I just need someone to show me where to start.
Thanks in advance for any assistance!
If you have systems or services you’re dependant so strongly, always have an backup / emergency access. 3rd party or self hosted.
My 5c but I think you agree.
Point being as a decades old it professional I see design more important as the detail implementation.
Indeed, I do agree but I’ve never done anything close to this magnitude so it is kind of intimidating for me. It is a learning process though!
I’d recommend not to go containerized but that can start a flame war. I would think it easier. But best to stick to the recommended beginner tutorial that someone else posted and go along from there.
Then ask questions on the way.
I hard agree with this. I would NEVER have wanted to start with containerized setups. I know how I am, I would have given up before I made it past the second LXC. Starting as a generalized 1 server does everything and then learning as you go is so much better for beginnings. Worst case scenario is they can run docker as the later on containerized setup and migrate to it. Or they can do what I did, start with a single server setup, moved everything onto a few drives a few years later once I was comfortable with how it is, nuked the main server and installed proxmox, and hate life learning how it works for 2 or 3 weeks.
Do i regret that change? No way in hell, but theres also no way I would recommend a fully compartmentalized or containerized setup to someone just starting out. It adds so many layers of complexity.
Thanks, you put it better than I can.
Native vs containerized really depends on what it is going to be doing tbh. If it’s just downloading and/or moving files around, containerized is fine. And having your docker-compose.yml files saved somewhere external will make future hardware upgrades/recovery much easier.
There is certainly some learning curve to figure out the quirks of a compose file, but the nice part is that most services will post an example compose file for you to edit as needed. And that means learning it is basically just a matter of reading the example files and figuring out what the different fields mean; yaml is extremely easy to read, even for someone who has never looked at it before. You may have some fringe cases that need a deeper dive, but the vast majority of setups are basically just a matter of “copy the example compose, edit the volumes as needed, and fuckin send it.”
Goddamned inflation. It used to be just 2 cents worth!
Ah crap that’s what I get for not being native English. But still, correct somehow.
I don’t believe inflation is a serious problem. But that is just my 9 cents.
What exactly do you want to do? Just have storage that you upload all your media to, which is also backed up somewhere else?
I think the very first step to building resiliency is to sign up for Proton’s cloud services. That will give you access to mail, both from Gmail via forwarding and a new inbox with a separate address. You’d also get a password manager and cloud storage. From there you can start self-hosting alternatives. Probably start with Immich as Google Photos is a big deal and it takes a ton of storage. Proton is a Swiss non-profit so the probability for enshitification is not nearly as high as with Google.
As soon as you have redundant storage, do a Google Takeout and download a full archive of your stuff. This feature may not be there for long given the current corporate climate.
Thanks, I will certainly look into this after I get some sort of basic understanding of the concepts at play.
That econdary drive I highly recommend you find a way to move that out of your house. For me I have a friend 8 hours away, we swap drives on occasion to keep each other’s backups in case of flood/fire/toddler or whatever other force of nature to save ourselves cloud backup costs
That’s a great idea. I’ve had a safety deposit box for years. I can just store it in there!
Safe deposit box is exactly the right size to hold a 3.5" HDD. Or several. I keep a backup Yubikey there too, because I love the physical token 2FA, but I’m pretty sure I’m going to lose it.
get your password situation squared away! every time i spin something new up i am grateful to have a pw manager to keep it all unique and maximum character limit
don’t even have to memorize the user of a lot of em
That’s the only thing that I do have taken care of! I basically immediately grabbed them out of Chrome and put them in KeePassXC on my PCs and KeePassDX for my Android.
Baby steps!
@MTZ @SidewaysHighways You may wish to disable all browser password managers, on all devices and use an alternate method of password management that suits your needs.
This is a cautionary tale on browser password managers (amongst other facepalms) that saw about millions of people’s personal details stolen - https://www.oaic.gov.au/__data/assets/pdf_file/0037/228979/Medibank-data-breach-alleged-timeline-infographic.pdf
While YUNO is a great way to get started, I strongly encourage you to understand basic concepts, like docker, and maybe try to run something outside of it for fun. While not even remotely the same thing since YUNO is just the OS and “app store”, you would be very similarly tied to that ecosystem the same way you are to Google now. Not to mean that YUNO would have any control over your stuff, but you would be dependent on them for what you can self host.
Yikes. Before you dip into any of the self-hosting, take and get a WD Gold drive - from Western Digital directly (wd.com) - do NOT go through Amazon or NewEgg or any third party merchant. Send in the warranty that goes with it and register the drive (this is for covering the off chance it’s a DOA unit) Then get a good quality enclosure to pop the drive into and take your time and back up EVERYTHING onto that new HD.
Don’t use an SSD.
You want a spinning platter drive, as this is backup only, so once it’s full with all of your content, it gets dated and labeled and popped into a drawer for safe keeping. If you have countless terabytes of data, get more drives and swap them into the enclosure, date and incrementally fill. A fine tip sharpie to note what’s on the drive is fine, or if you’re obsessively anal about it, make a spreadsheet with that info… If your drives are kept dry and stored with care they will last for DECADES…
The truth if being honest here - I’m a data hoarder and most of the stuff I’ve tucked away since I first came online (in 1999) is now on drives that I maybe spin up once a year. I used to have the notion that it was critical that all my shit was accessible all the time and I ended up dropping money on networked storage… and over time, realized that as long as I knew where the files were, DID have the most important stuff - family photos and scans - tucked away not only in long term storage, but on multiple drives in multiple machines, (home, work, laptop) it was okay not have it served up instantly.
Just reading your post made me go cold inside - I can only imagine what you were going through until it got sorted. From a bonafide old school data hoarder… Please, back your shit up locally. Use enterprise drives.
Then sort a self-hosting soultion.
Which software do you use for backups?
I do my backups manually.
As I have run unsuported Mac installs for the last 20 years, I started a long time ago, automatically partitioning my OS drives and making storage volumes to work off of.
The storage volume in the computer will have subfolders for the type of data - music, video, photos, etc.
When my storage volumes fill, I will pull my latest backup drive out of storage, hook it up then go into each storage subfolder, sort by date and add everything that’s newer than what’s in the backup drive. (which is actually how Apple’s Time Machine backups work - incrementally sorted by date - but I’ve had this method since the start, so I just stuck with it)
I just make sure to take note of how many files/folders I’m adding to the backup drive and note what it has at the start, then at the end, as a double-check of it all, before I clear the storage drive on the computer. (I did not do this and lost almost a years worth of music rips, waay back in 2003. Rebuilt the music I lost then iTunes threw a wobbler and lost the library for me. FML…)
The longest backup will ALWAYS be the initial one if you’re dealing with a first time backup. The rest, once you work out how to organize your files, is academic.
What I’ve found is that your tastes will change, you grab content you think you’ll want to hold onto forever… and then years later, you realize it’s low-bitrate, low-resolution, too pixellated… whatever… and you decide to delete it.
With the software doing the backups for you - it’s too easy to just let it rip and go have dinner while it works and you end up with files that you’d otherwise get rid of. Part of being a data hoarder is not keeping everything forever. There’s a ton of garbage online. Tastes change as you get older… You want to curate that shit so you can keep what’s most important - like family stuff.
And really good porn.
Several detailed, easy to understand and very good pieces of advice! Thank you! I have definitely saved your comment for referencing throughout this process!
Install Virtualbox (or some software to handle virtual machines).
Install Debian (or some other OS of your choice, I won’t judge if you prefer Windows).
Update your OS (apt update && apt upgrade -yon Debian).
Take a snapshot of your VM’s current state after updating. Saves a lot of time if you mess up or want a clean slate.Now you decide on what you want. Do you want to install n8n or Node-RED for automation? Do you want to use Immich for pictures? Paperless to save papers in a digital format? Audiobookshelf to listen on your books or podcasts? Jellyfin to stream your media? Set up a Minecraft or Factorio server?
Once you have decided on what you want to do, try to do it in your virtual machine.
Once you understand how to set it up and configure it to your liking, decide on how you want to host it. I took an office computer, added a few HDDs and replaced the case with a bigger one and it’s now my home server, but any old laptop will do. Just make sure to take backups.I used to have a Dell R710 and a virtual machine for each service I hosted, but I have moved to docker because it as simple as taking the often provided compose file, tweaking it a bit (where to store data etc) and running it with
docker compose up -d.It’s a great idea to run it in a VM first, because I’m bound to make mistakes along the way. Awesome advice! I’m definitely going to be referencing your comment throughout the process! Saved!
I had a Raspberry Pi once and wanted to move file from the current folder to some other folder. I typed
mv /* /path/to/folder/and move everything in the root directory and down to this other folder.EDIT: Meant to say that snapshots are cheap backups. I ended up reinstalled the OS.
Correct would have been dot slash:
mv ./* /path/to/folder/
Check out YUNOhost - it’s pre-configured for you and designed for beginners. Mine’s been running for about three years on a VPS with no problems and I had no previous experience with self-hosting.
Definitely keep your files backed up locally though. No server is invincible.
I will certainly look into that. I’ve never heard of YUNOhost but I’m going to give it a look soon!
Do it! It’s madness that YUNOhost is not more well-known, considering what it does.
This post inspired me to audit my firewall rules, had put it off for too long
I wanna say thank you for making this post OP. I’ve got a spare laptop that I want to try to turn into my own cloud server but I find the endeavour similarly hard as well. I’ll be looking at the tips in the comments. Good luck OP!
Good luck to you, as well friendly stranger!
I’m 100% self taught & was in exactly the same place. I’d never used Linux before I got my first Pi. I spent a bit of time trying to familiarise myself with & made some notes regarding command line (notes I still rely on).
There’a so many ways to achieve the goal, you’ll eventually find a way that works for you. My personal preference was Docker/Docker Compose deployed via Portainer.
Even that was confusing. Until I found this excellent video on how to read Docker requirements & apply them step by step into Portainer. He explains slowly & methodically exactly what he is doing & why.
Portainer is a method of handling Docker stacks/containers via a web UI. Both Docker & Portainer are simple to install.
It’s easier to use Docker Compose files and/or .env (environment variable) files (both are even simpler to deploy through Portainer) but this video taught me what was going on & gave me confidence to have a go. What attracted me to Docker is you can easily remove stacks/containers if/when you make a mess rather than wiping the drive & start again, which is how I went about things initially.
This gave me the tools to set up Nginx Proxy Manager & I never looked back.
As you’ve realised, a robust backup solution is essential (plus off site backup for particularly important stuff) as things will inevitably go wrong along the way (I see Borg, Restic mentioned often, I went for Kopia).
I can’t recommend highly enough making detailed notes along the way, I rely on Joplin.
If you start using Docker, dont fall into the trap of using the “latest” tag. If you know the version number you’re running its far easier to re-deploy if an update is bad.
Enjoy your new time consuming, teeth gnashingly frustrating …and yet rewarding hobby 👍
Hi @MTZ , #selfhosting could be a move in the right direction for you. I started managing my own servers over 10 years ago, locally, from my home, later VPS and finally again from my home. Eventually I moved toward @yunohost - it simplifies a lot of things! I documented some my experiences at https://wasi.ovh
Start small: setup file/photo sync (@nextcloud), calendars and contacts and gradually start adding data from old backups once you feel comfortable.
Have fun and good luck :)start small … nextcloud
As the result of a single misconfigured security setting on my Android, I was locked out of my Google Account on my phone AND all of my PCs.
Just a heads up on what you are getting yourself into, if you fuck up your self hosted setup badly enough there is no recovery.
That isn’t necessarily intended to scare you off from self hosting, just that the first and most important lesson to learn is to have a good system of backups that are backed up automatically, are easy to recover from, and are separated enough from other copies of the data that if something goes terribly wrong one copy will survive.
Thanks for the advice. Currently, I have a backup of all of my important data on a high capacity HDD that is completely disconnected from any devices. There is no real way to automate backups with that setup, but it’s what I am working with at the moment.
I’d recommend setting your sights on the 3-2-1 rule. 3 copies of your data, 2 different mediums, 1 off-site. Hetzner Storage Box is a good cheap offsite option.
To piggy back off of Arcayne’s comment
I’d recommend setting your sights on the 3-2-1 rule. 3 copies of your data, 2 different mediums, 1 off-site. Hetzner Storage Box is a good cheap offsite option.
I don’t backup everything to a 3-2-1. I at least have everything on a 2-1 backup scheme. Two copies, one offline. My personal photos and videos are on a proper 3-2-1. I can redownload my movies and music. I can’t retake pictures from my childhood. It saves a lot of money doing it that way. My personal media is less than 1 TB, whereas my whole collection is many TB and would be prohibitively expensive to 3-2-1.
As someone who went through this after trumps 2nd term and power grabs i can give you my process:
- angrily unsubscribe all big tech subscriptions
- make a protonmail and tutamail account, realize I like proton suote more and decide to subscribe
- transfer all passwords to proton suite
- download all photos and other from cloud to an external drive. TURNS OUT THIS TAKES SEVERAL DAYS WTF
- angrily order a rasp-pi and an external SSD
- use step by step tutorials from pimylifeup to install docker and immich. Fall in love
- gradually (via help of google and GPTs) become confident enough with command line to start managing the server headless over SSH
Fast forward 6 months: My router is now running OpenWRT. With a few necessary exceptions my network access is always through ProtonVPN. My external devices are connected via wireguard to the router when not on home wifi and only after that reach the www. I have 24/7 access to my services from everywhere. My main server is now an old office mini pc running about 10 services. Im using borg for nightly snapshots(its a bit like apple time machine) and after that everything is backed up to another server at a friends house via rsync and ssh. I have a third mini computer whose purpose is to be my tv’s UI with access to services like the national broadcasts web ui and my own jellyfin and invidious (adless youtube client) The tv does not have an internet connection anymore. I even made a custom land page that automatically opens full screen in a browser when open my tv.
The point is: this builds gradually and you have fun doing it. …until it breaks :D The most painful parts involved networking so you can settle for LAN only at first to keep things simple
how well does invidious work on the TV? how easy are the controls?
I use a small wireless logitech keyboard-mousepad so it works very well. I had to make exceptions in the router for googles video severs to bypass the vpn though.
yeah not all invidiouses allow proxying the video, and at some it used to be off by default
Re-investing in a new platform full of tools (Proton suite) isn’t in my opinion a rational answer. My answer is self host vaultwarden, self host your file storage, and choose between Proton and Tuta for mail, and use your own domain name so you can take your email address with you should you move.
In my opinion No-one should ever store any form of personal data implicitly on someone else’s computer.
Oh i have vaultwarden as well nowadays
When you’re connecting to your local network, say from outside your home, you connect through proton vpn? I’m wanting to self host, but I thought I would have to switch between proton and whatever VPN the local machine is using? Say immich for instance
My phone is on a wireguard tunnel into my router which puts my wireguard vpn in the same forewall zone as my home LAN. Internet access is routed through the tunnel and then through another tunnel to protonvpn and from there to the www. It was a bit elaborate to set up but it works. Wouldnt really recommend the setup for everyone, it was a bit of a pain in the ass to get working. I used Openwrt and policy based routimg plus wireguard for the tunnels into and put of the router.
