Self Hosting: It's like being a prepper for the cloudpocalypse

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > Self Hosting: It's like being a prepper for the cloudpocalypse

Hughlander: May 11, 2005

Motronic posted:

I'm still using NextCloud and their terrible photo browser on a computer and "Les Pas" on my phone. It's not a great solution.

I haven't tried immich in a while. It's very promising, but very incomplete. It's probably worth a try for you )(and for me to see how far they've gotten since the last time I tried it).

The more mature one is PhotoPrism. Which bafflingly doesn't have any concept of "users". There were also some other annoyances that may have risen to the level of deal killer for me, but I don't recall them. It may very well work for you - we've all got different requirements.

NextCloud background upload of photos on iOS is really crap everytime I looked at it. Like it never finished the initial sync from an iPhone. I'd love something that 'just worked' I've taken to just having my wife on a spare Apple Mini that gets iCloud drive photos + time machine to my NAS but it still sucks.

# ¿ Oct 4, 2023 19:28

Adbot: ADBOT LOVES YOU

# ¿ May 14, 2024 05:01

Hughlander: May 11, 2005

Kivi posted:

Is there any container thingy that would give me and my friends a easy to use shareable GIF/funny pics folder? Like just box with possibility to add GIFs and stuff and then post them on forums. It should have some sort of login for uploads, but public browseable folder for guests.

I used pinry for that. Basically self hosted Pinterest

# ¿ Oct 8, 2023 17:38

Hughlander: May 11, 2005

e.pilot posted:

if a friend and I wanted to do offsite backups for each other what would be the best way to go about that?

SFTP?

A VPN tunnel that only points at an FTP?

Install sanoid/syncoid and run it over ssh + tailscale

# ¿ Oct 9, 2023 02:32

Hughlander: May 11, 2005

Flyndre posted:

I've finally got around to set up Radarr, Sonarr, Prowlarr, Jellyfin and Jellyseerr on my Synology NAS, and it seems to work well.

One question: What is the workflow supposed to be like, when I add an older TV-show to Sonarr where I also want to download older seasons? For some of the shows it only finds the newest episodes. As far as I've understood, this is intended behaviour since it's based on RSS-feeds from the index providers. Do I then need to grab older episodes manually (which I've seen that I can do in the GUI), or is there a cleverer solution?

I'd just double check that the emblem that shows what's being monitored has the previous seasons being monitored. Then as long as you have a usenet index it should find it. You can also tell it at time you add the show if it should look for previous seasons or not so make sure that's on. I'd stop by the Usenet thread for more questions though.

# ¿ Oct 14, 2023 15:51

Hughlander: May 11, 2005

Motronic posted:

I recently had to upgrade the base OS (well, replace) and reinstall nextcloud because of.....I don't recall...php? Whatever dependence wasn't being satisfied to continue upgrading. I moved the old install which was three major releases behind and went through the upgrade processes like 4 times? I had the same problem I always have: timeout of the webgui during backup. You can wait that out and continue the process and it completed fine every time.

What kinds of stuff is breaking for you or others?

Maybe it's a poor expectation of mine, but I don't think I've ever had a docker pull nextcloud && docker restart nextcloud actually work. To me table stakes for a container is that it handles the janitor work in the background.

# ¿ Oct 16, 2023 17:07

Hughlander: May 11, 2005

Mr. Crow posted:

You have to specify a major version as nextcloud only supports major version upgrades. So nextcloud:26-apache and you can just pull and restart to get the latest point release. Pull 27-apache when you're ready to pull the next major version and so on.

Its definitely the worst OSS community out there though as far as quality and usability and reliability, not sure why. PHP is rear end maybe? Never used it, no desire to either, maybe thats true for lots of other devs who might otherwise be willing to contribute and fix stuff.

Yah I checked my notes I was just running with:
image: lscr.io/linuxserver/nextcloud

Sloppy on my part.

# ¿ Oct 16, 2023 20:44

Hughlander: May 11, 2005

Sub Rosa posted:

Built a new NAS, running Unraid. Trying to get a handle on how I want to buildout dockers.

Can someone give me the current landscape in regards to Nginx vs Traefick? Swag vs Nginix Proxy Manger?

I mean traefik is made for that...

I set up a .env and basically add a few lines to docker-compose
labels:
traefik.enable: true
traefik.http.routers.grocy.rule: "Host(`grocy.${DOMAIN}`)"

AFAIK that's all it takes for the domain to get brought in. However, I set up a auth middleware that if you're external to local network you need to login with Organizr (Have access to my plex server.) and based on your level in that you may or may not have access:
labels:
traefik.enable: true
traefik.http.routers.grocy.rule: "Host(`grocy.${DOMAIN}`)"
traefik.http.routers.grocy.tls: true
traefik.http.routers.grocy.middlewares: "secured-admin"
traefik.http.routers.grocy.priority: 99
traefik.http.routers.grocy2.rule: "Host(`grocy.${DOMAIN}`) && ${PRIVATE_IP}"
traefik.http.routers.grocy2.tls: true
traefik.http.routers.grocy2.middlewares: "secured-local"
traefik.http.routers.grocy2.priority: 100

where PRIVATE_IP is defined in the .env along with DOMAIN:
PRIVATE_IP=HeadersRegexp(`X-Real-Ip`, `(^127\.)|(^10\.)|(^172\.1[6-9]\.)|(^172\.2[0-9]\.)|(^172\.3[0-1]\.)|(^192\.168\.)`)

I just copy grocy's docker-compose to any new one and search/replace grocy with newname. There's also a secured-user defined for things I want other folks to use.

# ¿ Nov 13, 2023 19:55

Hughlander: May 11, 2005

cruft posted:

This is a question about SMTP.

People at work keep emailing my personal address. I forward those emails to work so I can reply from the right address, and they get stuck in some kind of weird Microsoft quarantine nobody knows about. This almost prevented me from getting hired at this place, because they didn't know about the quarantine, and thought I was ignoring them.

This is probably happening elsewhere.

I asked around and they said it's probably happening because I don't have DMARC set up. I never set that up because it looks like I need to set up DKIM first. I never set up DKIM, because my vanity domain is just going through Gmail, and it looks like DKIM requires me paying far out the rear end for Google Workspace.

It's felt for a while like running email to my own domain was a hobby I would get squeezed out of due to anti-spam measures ramping up the maintenance costs beyond my ability to manage in my spare time. Maybe this is that point. Or maybe I'm missing something obvious! Any opinions?

As you point out the anti-spam keeps ramping up. Some places apply a huge negative value to anything that looks like a Cloud IP or a consumer ISP IP. Even if you jump through every hoop you still could be quarantined because they think you're faking it. But that shouldn't happen if you're coming from Gmail. I'm surprised you aren't considered grandfathered into the old Workspaces? Or did you just never use it for that?

More I think about it the odder this sounds. If it's all through gmail it 'should just work' I've never heard of a problem with that with my half dozen or so vanity domains that I use. Can you explain a bit more about your workflow? How are you forwarding the email to work then not seeing a quarantine?

Is this what's happening:
A Mails Home
You forward to Work
Work mails A
Email get's quarnateed at A?

# ¿ Nov 30, 2023 00:02

Hughlander: May 11, 2005

cruft posted:

Okay, sure. Let's pretend my domain is "example.org".

I have gmail set up so that I can send email out as cruft@example.org

I've got a TXT record on example.org saying "v=spf1 include:_spf.google.com ~all"

I have a TXT record on _DMARC.example.org saying "v=DMARC1; p=none; rua=mailto:cruft@example.org; ruf=mailto:cruft@example.org; fo=s; aspf=s"

Mail goes out with "From: Cruft McCruftykins <cruft@example.org>"

But the SMTP envelope says "MAIL FROM: unprofessional@gmail.com"

According to the postmaster at work, the big problems are that the SMTP envelope doesn't match the From header, and that DMARC does not provide DKIM.

Also Google is on a couple blacklists. I'm sure they have an entire team who pretty much tries to stay off blacklists all day

e: oh, hey, to answer your actual question:

a@spumco.com mails cruft@example.org

I reply to it and Cc cruft@spumco.com, with the intention of replying from work and removing cruft@example.com, so it threads for everybody

It gets stuck in an obscure quarantine that nobody at spumco has ever heard of

ee: I found some more information. It looks like changing the SPF failure policy to hard fail (-all), and the DMARC policy to p=reject might help. I'll try that and report back in 6 hours when my TXT record expires

Got it. Now I need to see what mine does for that same case...

So for a source without DMARC set up at all:

quote:

Received-SPF: pass (domain of gmail.com designates 209.85.167.49 as permitted sender)
Authentication-Results: atlas217.free.mail.bf1.yahoo.com;
dkim=pass header.i=@gmail-com.20230601.gappssmtp.com header.s=20230601;
spf=pass smtp.mailfrom=gmail.com;
dmarc=unknown header.from=DOMAIN2.com;
...
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail-com.20230601.gappssmtp.com; s=20230601; t=1701302982; x=1701907782; darn=yahoo.com;
h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
:date:message-id:reply-to;
bh=hE0tJudIEQ/CwWwSC+EseFqI2NmWsaYUhsG+i5uoB2c=;
b=bu3T9uo1754CSukR3KfcXPhyx0SH+USLquHrR+Ygu/qUCqP+sLl4rdHniS4liecVkS
fLhiEBqa7J7EgTIlSsjMYCM0IGAC6t2pFXTFtr0EIsX+KI7xSxv3SYLDEWqZFG/hrWep
f4ZjE5uDYbywcUKUtpywo0w0FR5NNH/afwTWyoHknbFIHO17EAafKvivg+4z6KiylSb0
+56k/p7bzG45ekPvqkAM5/rW3fXZ7XiwmVKIvisYC0QHI7HzJbVTX+s/p4H8g5Z2DoM0
WZu0vIdtKeJSmmBtK2qFLVg6DwGmqmt/Bgj+3arA/8pPAZYVMmkcYoN4u7JpBt/MDzmV
7bEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1701302982; x=1701907782;
h=to:subject:message-id:date:from:mime-version:x-gm-message-state
:from:to:cc:subject:date:message-id:reply-to;
bh=hE0tJudIEQ/CwWwSC+EseFqI2NmWsaYUhsG+i5uoB2c=;
b=VynketkmJHTfULh59HnxsGffCUllE5rkOb860sqz+7d4yxwJSpZo7cE9JZXR1h6ugh
fmqU3ghUNuRrCyUPgBOcObqaUmxmBtiC1RKv8PB0r46Xz+JgqPVe6dm0YmwV+iCxnZhb
PIndfahhI4NVxRrGIM1xA3oQpi8PREnnt9woy3I4iorun0ZOk0waZ3NBukn5uR2EQsQQ
3yW558+3RXSnlC7gOx3WaesBr1U3mwe00Tlcvt0XclbhdzhkXHw+Civx0myVNIb83UjE
5He2F9TfKxp1sPJMTntqiMOwoAV8NjcLSjOysHH1zcuLOQeAtccPZEbXSPp9ZqhdJIqn
/bxw==

So google is giving it a DKIM signing even though dmarc says 'unknown header'

Now what's concerning to me is the domain I *THOUGHT* I had DKIM set up for says:

quote:

Received-SPF: pass (domain of gmail.com designates 209.85.167.43 as permitted sender)
Authentication-Results: atlas108.free.mail.ne1.yahoo.com;
dkim=unknown;
spf=pass smtp.mailfrom=gmail.com;
dmarc=unknown header.from=DOMAIN1;
...
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1701302939; x=1701907739;
h=to:subject:message-id:date:from:mime-version:x-gm-message-state
:from:to:cc:subject:date:message-id:reply-to;
bh=7BHIuqT59UkXKgy4Zj86G2qTrcBA1onYiW0vSRo9dQk=;
b=BPQaQSqoNUi9r1+ClaBYU2fszBKfNtBcVATh5/vo9Giu9gN0SByN3yK5Wl0gf5Ma5m
YQRFAqUq9/S45I6DJK6dkGHkIk/h1jTzTgtdSqpjRX8Mts1/ksU8F07RNMvpKBT6rU0m
1417Avlmx/RMc9dPlI/hSTt1XsXgsEIf0ibI/soRUA+sXtwXNqsr/6hPHyEl7YbGroZ0
QEqcJipE+3D5MKSi88h9+/Ib0jOU+wFNNBVScZCQggDmxjxTumCWJ+VnCx0hfoGAWw74
Jg2Up6Dl9+EypVw+oyJWr1GUhZyR5maJRimr1HlaiCpPCqNQENzt5lYE5Z0snaZ8a8nw
Z4lg==

So only the X-Google-DKIM-Signature, and not the DKIM-Signature, and dkim of unknown in the Auth results block.

Great now you gave me a mystery.

# ¿ Nov 30, 2023 01:17

Hughlander: May 11, 2005

cruft posted:

You're routing through Yahoo?

Nah I mailed a yahoo account since it was the only one I could think of that wasn't hosted by google or my work where I don't want my work knowing my personal domains.

# ¿ Nov 30, 2023 01:31

Hughlander: May 11, 2005

Heck Yes! Loam! posted:

This just reinforces why you don't self host email

Except this really isn't self hosted. In both cases being discussed I believe google is hosting all the mail.

# ¿ Nov 30, 2023 02:03

Hughlander: May 11, 2005

THF13 posted:

Dockge looks to be an excellent tool for managing the various docker based selfhosted apps/services while using regular ol' Docker compose.

You get a pretty basic WebUI where you can deploy the stacks, edit existing compose.yml files, and start/stop/update/restart your services. You also get a web console that can run commands on the host or inside individual containers. You can see the progress of containers being pulled/started and view the logs of running containers.

Behind the scenes you specify a "stacks" folder, and Dockge creates a subfolder inside there for each service, where the associated compose.yaml file for it will live and it assumes you will place application configs and things.

What I like about this is just using Docker compose, it's not taking over anything or using it's own weird thing. You can create and deploy a container inside that stacks folder however you want and manage it with this, or edit a container this created using whatever program and tools you'd like.

Having used Unraid for a long time, I've gotten really used to having basic management of containers being just 1 or 2 clicks away, and found that very useful when dealing with basic self hosted services which are typically single instances running on singular hardware, and not dealing at all with swarms or scaling or everything enterprise docker related, and this gives me that.
I've tried Portainer and it's fine, but always felt over-engineered and clunky.

It's open source and from the same developer as uptime-kuma, with a similar UI to that.
https://www.youtube.com/watch?v=AWAlOQeNpgU

That looks cool, does it support either fragments of compose files and/or .env files?

My workflow for any new docker is basically:

code:

mkdir NEWPROJECT
cd NEWPROJECT
ln -s ../.env .
cp ../grocy/docker-compose.yml .
vi docker-compose.yml
:1,$s/grocy/NEWPROJECT/g
:wq
docker volume create -d zfs NEWPROJECT_config
docker compose up -d

Where grocy/docker-compose.yml looks like:

code:

version: '2'
networks:
  jefferson_default:
    external: true
volumes:
  grocy_config:
    external: true
services:
  grocy:
    image: lscr.io/linuxserver/grocy
    container_name: grocy
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
    volumes:
      - grocy_config:/config
    restart: unless-stopped
    networks:
      - jefferson_default
    labels:
      traefik.enable: true
      traefik.http.routers.grocy.rule: "Host(`grocy.${DOMAIN}`)"
      traefik.http.routers.grocy.tls: true
      traefik.http.routers.grocy.middlewares: "secured-admin"
      traefik.http.routers.grocy.priority: 99
      traefik.http.routers.grocy2.rule: "Host(`grocy.${DOMAIN}`) && ${PRIVATE_IP}"
      traefik.http.routers.grocy2.tls: true
      traefik.http.routers.grocy2.middlewares: "secured-local"
      traefik.http.routers.grocy2.priority: 100

That's a lot of boiler plate for setting up traefik on a shared network, using a zfs volume for config, etc...

# ¿ Jan 8, 2024 19:33

Hughlander: May 11, 2005

cruft posted:

Friend, have you heard about Kubernetes?

I think the difference here that your process is "go hack a couple files and kick it up with docker", which is cool. Docker Swarm and k8s are "you are running a massive installation and need to be able to recover from a plane crashing into the data center in under 4 hours".

For homelabs, your process is just fine, and arguably a better use of hobby time.

? Weren't we basically talking about a more personal version of portainer though? Dockage hardly seems to be multi region k8s level unless I missed something in the video.

# ¿ Jan 8, 2024 19:45

Hughlander: May 11, 2005

cruft posted:

Does anyone else ever get the feeling it's the same three dozen goons in every thread they've bookmarked?

I mean I do think that naturally there's going to be huge over lap towards:

I like watching media in Plex
That I store on my NAS
that I selfhost the software
to get things from Usenet

And you're going to see the same people talking in those threads

# ¿ Jan 12, 2024 18:54

Hughlander: May 11, 2005

Corb3t posted:

Why do people self host e-mail? For privacy purposes? Becuase it's nearly free? I have to imagine most self hosters own a domain or two with really cheap shared hosting plan. I don't pay much on Namecheap.

I did for the longest time just because it's what I always did.

I started an ISP in college, when I sold it, I colocated a half rack wherever I happened to be living, that became rented some VMs eventually, until fiber made it all self hosted. At some point though I realized it just wasn't worthwhile with free Google Apps For Domains.

# ¿ Jan 14, 2024 19:02

Adbot: ADBOT LOVES YOU

# ¿ May 14, 2024 05:01

Hughlander: May 11, 2005

Resdfru posted:

I run trillium for notes but I've never really loved it. I saw notion in the play store and just from the screenshots it looks awesome. After some digging it looks like the most similar things I can self host are

Appflowy
https://github.com/AppFlowy-IO/AppFlowy

Outline
https://github.com/outline/outline

Anytype
https://tech.anytype.io/how-to/self-hosting

Anyone got any experience with em?

There's a low volume thread about notes. https://forums.somethingawful.com/showthread.php?threadid=3990615 I'm personally part of the Obsidian cult. Electron client with local markdown files that you then sync to other locally held obsidian instances. I run a copy in a docker container with vnc in case I really don't have access to my desktop, my mac, my ipad, my iphone, etc...

# ¿ Feb 3, 2024 18:45

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > Self Hosting: It's like being a prepper for the cloudpocalypse