Selfhosted

I'm using CheckMK to monitor my hypervisor, physical hardware like disks, CPU etc. and SNMP-capable hardware like my pfSense firewall via a CheckMK instance in docker. It either works in docker or on a few different linux based OS like ubuntu and debian (see CheckMK download page).

There's a free and open source version (called raw edition, GitHub Link) which I am using. It comes with a lot of checks / plugins for monitoring stuff out of the box and if there's something it doesn't ship, you can easily create your own check in whatever language your server is capable of executing a binary of. Or you could look up if there's a user-contributed plugin on the official CheckMK Exchange Platform.

The whole configuration of this is based on rules with a lot of predefined rules and sane defaults already set.

To have an example for your use-case: You can monitor docker-logfiles and let CheckMK warn you, if specific keywords are or are not in a logfile. You will then be able to view the offending lines in the monitoring UI.

Why do I use this?

• We use it at work
• FOSS
• docker makes updating this easy
• can send mails, teams notifications, ...
• very customizable and expandable

my docker compose file

# docker-compose.yml

services:
  monitoring:
    image: checkmk/check-mk-raw:2.4.0-latest
    container_name: monitoring
    restart: unless-stopped
    environment:
      - CMK_PASSWORD=changeme
    ports:
      # WEB UI port
      - "5000:5000"
      # agent communication port
      - "8000:8000"
      # used for SNMP
      - "162:162/udp"
      - "514:514/tcp"
      - "514:514/udp"
    volumes:
      - "./monitoring:/omd/sites"
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env

Start off simple, use something like uptime-kuma just to check your services are available - takes minutes to set up and can send you notifications when something goes down. It can plug into docker directly to check if a container is up, as well as perform HTTP checks that the service is responding, plus some other cool stuff.

(Side note, I set up ntfy to handle notifications and it's great! Another solid recommendation but you can use discord web hooks or whatever as well)

The other options described here are good for gathering and visualising data, but it takes quite a bit to set them up and even more to configure the right kinds of alerts to notify you when something is wrong. A simple "is this docker container running" check or a "does this respond with a http 200" check gets you like 95% the way there.

Librenms

I've recently switched to Beszel and so far it's been great.

I know these projects cockpit, netdata and grafana + prometheus.

Prometheus+Grafana is my go-to. You can also add Drilldown plugin to Grafana and it will monitor Docker logs.

Prometheus is incredibly versatile and widely used, so a lot of projects support metrics export to it, so IMO is your safest blind bet.

For even more custom behaviour, you can create very simple Python scripts that send data to Prometheus.

It's not primarily made for monitoring, but Dockhand has a lot of great monitoring for your whole host and for individual containers.

There are a lot of server monitoring softwares out there, but Netdata gives you just about every metric I'd ever want to look at. Can be deployed using Docker (https://learn.netdata.cloud/docs/netdata-agent/installation/docker).

The one thing I do when linking to Netdata in whatever dashboard you are running, I use the v3 switch at the end of the url thusly:

https://netdata.mycoolserver.com/v3

That way you don't have to log in to the mother ship to view your metrics.

ETA: mobile app available

SigNoz or Uptrace are alternatives to something like DataDog, which is the route you want to go versus checking each individual machine.

You could also just use Prometheus+Grafana and build your own monitoring dashboards and alerts that way, but will be a bit more manual at first.