
You SSH into a server, start a Python script with nohup, and cross your fingers. Three days later, you realize it died when the system rebooted for a security patch. Sound familiar?
Every developer hits this wall eventually. You write a script that needs to run continuously — a Discord bot, a data scraper, a health check endpoint — and the moment you log out or the server restarts, it’s gone. You need it to start on boot, restart when it crashes, and run quietly in the background like the pros do it.
That’s exactly what systemd service files are for. And once you learn them, you’ll never go back to cron-hackery or screen sessions.
I’ve been managing Linux servers for over a decade, and systemd is one of those things that feels intimidating until you write your first service file. Then it clicks — and suddenly every long-running script you write becomes a proper, managed service. Let me walk you through it.
What systemd Actually Is
systemd is the init system on virtually every modern Linux distribution — Ubuntu, Debian, Fedora, Arch, RHEL, you name it. It’s the first process the kernel starts (PID 1), and it’s responsible for launching and managing everything else: networking, logging, cron, SSH, your display manager.
But for our purposes, the key concept is this: systemd can manage your scripts the same way it manages nginx or PostgreSQL. A “service” in systemd is just a unit file — a small text configuration that tells systemd what to run, how to run it, and when to restart it.
You keep those unit files in /etc/systemd/system/ (for system-level services) or ~/.config/systemd/user/ (for user-level services that don’t need root).
The Real Problem We’re Solving
Let’s make this concrete. Imagine you’ve written a Python script that monitors disk usage across your servers and sends you a Telegram alert when things get tight. You need it to:
- Start automatically when the server boots
- Keep running even after you log out
- Restart if it crashes (scripts crash — that’s reality)
- Log its output somewhere you can check later
You could do this with a cron @reboot entry and a wrapper script. You could also duct-tape your muffler to the car frame, but there’s a better way.
Step 1: Write the Script (Keep It Simple)
Create a shell script that does one thing and logs its output. I’m using a bash example because it’s the lowest common denominator — no Python, no Node, just /bin/bash.
#!/bin/bash
# /usr/local/bin/disk-monitor.sh
THRESHOLD=90
LOG_FILE="/var/log/disk-monitor.log"
while true; do
USAGE=$(df / | awk 'NR==2 {print $5}' | sed 's/%//')
if [ "$USAGE" -gt "$THRESHOLD" ]; then
echo "$(date): WARNING — Disk usage at ${USAGE}%" >> "$LOG_FILE"
# Send your Telegram/webhook alert here
fi
sleep 300 # Check every 5 minutes
done
Make it executable:
sudo chmod +x /usr/local/bin/disk-monitor.sh
Test it manually first — always. Run it in the foreground for a minute and make sure it writes to the log file. A systemd service wrapping a broken script just gives you a reliably broken service.
Step 2: Write the systemd Unit File
This is the heart of the tutorial. Create a file at /etc/systemd/system/disk-monitor.service:
[Unit]
Description=Disk Usage Monitor
After=network.target
[Service]
Type=simple
ExecStart=/usr/local/bin/disk-monitor.sh
Restart=on-failure
RestartSec=10s
User=nobody
Group=nogroup
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
Let me break down what each section and directive actually does — because copying and pasting without understanding is how you end up debugging at 2 AM.
[Unit] — Metadata and Ordering
Description= is cosmetic but important. It’s what shows up when you run systemctl status or systemctl list-units. Make it human-readable — “Disk Usage Monitor” is better than “diskmon v2 final final2.sh”.
After=network.target tells systemd: “don’t start this service until the network is up.” If your script makes HTTP requests (Telegram API, webhooks), this prevents a race condition on boot where your script starts before the network stack is ready and immediately fails. Common alternatives include After=mysqld.service or After=nginx.service if your script depends on a database or web server.
[Service] — What to Run and How
Type=simple is the default and most common type. systemd considers the service started as soon as it forks the process. For scripts that run in an infinite loop (like our disk monitor), this is exactly what you want. If you’re wrapping a daemon that forks and backgrounds itself, use Type=forking instead.
ExecStart= is the command systemd runs to start your service. It must be an absolute path — relative paths won’t work because systemd doesn’t use your shell’s $PATH.
Restart=on-failure is the directive that changes your life. Options:
- no — Never restart (default)
- on-failure — Restart only if the process exits with a non-zero code or is killed by a signal (most common choice for scripts)
- always — Restart no matter what, even if you run
systemctl stop(rarely what you want) - on-abnormal — Restart on signal crashes only
RestartSec=10s adds a 10-second delay before restarting. This prevents a “thundering restart” where a broken script crashes, restarts immediately, crashes again, and floods your logs. For scripts that might flap (network-dependent scrapers, API callers), bump this higher — 30s or even 60s.
User= and Group= are security fundamentals. Never run your service as root unless you absolutely have to. Create a dedicated system user with sudo useradd -r -s /usr/sbin/nologin diskmon and use that. Our example uses nobody:nogroup — the least-privileged account on the system, suitable for scripts that just need to read files and write to logs.
StandardOutput=journal / StandardError=journal sends stdout and stderr to the systemd journal — viewable with journalctl -u disk-monitor. Without this, output disappears into the void. You can also use StandardOutput=file:/var/log/myservice.log for a traditional log file, but journald is more powerful (time-based filtering, rotation, and you’re already paying the overhead of running it).
[Install] — When to Start
WantedBy=multi-user.target tells systemd to start this service when the system reaches multi-user mode (i.e., normal boot, right after networking). For user services, use WantedBy=default.target instead. This only takes effect after you enable the service — it creates a symbolic link that tells systemd to start your unit as part of that target.
Step 3: Install and Enable
You’ve written the file. Now tell systemd about it:
# Reload systemd's configuration
sudo systemctl daemon-reload
# Start the service immediately
sudo systemctl start disk-monitor
# Enable it to start automatically on boot
sudo systemctl enable disk-monitor
The order matters. daemon-reload picks up changes to unit files — skip it, and systemd is still working with cached state. enable without daemon-reload first is a waste of keystrokes.
You can combine start and enable into one line (but I prefer the two-step for clarity):
sudo systemctl enable --now disk-monitor
Step 4: Verify It Actually Works
This is the part most tutorials skip. Don’t be that person who deploys a service at 5 PM on Friday and checks on Tuesday.
# Check the service status
sudo systemctl status disk-monitor
# Look for: Active: active (running)
# Main PID: 12345 (disk-monitor.sh)
# Check the journal output
sudo journalctl -u disk-monitor -n 20 --no-pager
# Confirm it's enabled to start on boot
systemctl is-enabled disk-monitor # Should print "enabled"
# Simulate a crash to test restart behavior
sudo kill -9 $(systemctl show -p MainPID --value disk-monitor)
sleep 12 # Wait for RestartSec
systemctl is-active disk-monitor # Should still print "active"
The last test is the real one — actually kill the process and verify it comes back. If it doesn’t, check your Restart= setting and make sure the process PID actually changed.
User-Level Services (No sudo Required)
Not everything needs root. If you’re running a bot from your home directory or a development script that only affects your user, use systemd’s user mode:
# Store unit files here (create the directory if needed)
mkdir -p ~/.config/systemd/user/
# Write your service file at:
# ~/.config/systemd/user/my-bot.service
# Same unit file format, but without User=/Group= directives
# (it runs as you by default)
systemctl --user daemon-reload
systemctl --user enable --now my-bot
# To survive logout, enable lingering:
sudo loginctl enable-linger $USER
The --user flag is the only difference. One critical detail: user services stop when you log out by default. The enable-linger command tells systemd to keep your user session alive even when you’re not logged in — essential for bots and background workers.
systemd Timers: Cron, But Better
Once you’re comfortable with service files, timers are the natural next step. Instead of cron’s cryptic syntax, systemd timers use a clean declarative format:
# /etc/systemd/system/cleanup.timer
[Unit]
Description=Run cleanup every day at 3 AM
[Timer]
OnCalendar=daily
OnCalendar=*-*-* 03:00:00
Persistent=true
[Install]
WantedBy=timers.target
Pair it with a service file (cleanup.service), and now you have scheduled jobs with all the benefits of systemd: logging to journald, email-on-failure, dependency management, and the ability to list all scheduled tasks with systemctl list-timers.
The Persistent=true option is the killer feature cron doesn’t have — if the server was down at 3 AM, the timer fires immediately on the next boot instead of skipping the missed run.
Common Pitfalls and How to Avoid Them
I’ve made every one of these mistakes so you don’t have to:
“My service starts but dies immediately.” Check your script’s exit code — if it exits with 0 and Restart=on-failure, systemd won’t restart it. For infinite-loop scripts, make sure the loop never exits. Run the script manually first: /usr/local/bin/disk-monitor.sh.
“systemctl status says ‘inactive (dead)’ but no errors.” Your Type= is probably wrong. Type=simple (the default) works for most scripts. If your service daemonizes itself (forks to background), you need Type=forking and PIDFile=.
“It works when I run it manually but fails under systemd.” Environment mismatch. systemd runs with a minimal environment — no $PATH, no $HOME, no shell aliases. Use absolute paths in your scripts. If your script needs environment variables, declare them with Environment= or EnvironmentFile= in the [Service] section.
“journalctl shows ‘command not found.'” Same issue — your script is using a relative command name. Always use the full path or set Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin in your unit file.
“Restart=on-failure isn’t restarting my service.” If your process exits with code 0 (success), on-failure doesn’t restart it. This is by design. If your script runs once and exits successfully, use a timer, not a service. Or switch to Restart=always if you genuinely need it.
What You Can Do With This Now
Once systemd service files click, you start seeing opportunities everywhere:
- Telegram/Discord bots — Run them as services instead of tmux sessions. If you’ve read my tmux guide, you know how useful tmux is for interactive work — but background bots belong in systemd.
- Webhook listeners — Small Flask or FastAPI apps that receive webhooks and process them asynchronously need to stay up 24/7.
- Data scrapers and ETL pipelines — A service that pulls data from an API every hour and writes to a database. Pair it with a timer for scheduling.
- Health check responders — A tiny HTTP server that returns 200 OK for your load balancer’s health checks.
- File watchers — Monitor a directory for new uploads and trigger processing pipelines.
Every single one of these is better as a systemd service than a screen session, a nohup hack, or a cron job that runs once and hopes nothing crashed in between.
The Bigger Picture
systemd is part of a larger Linux server administration toolkit. If you’re managing a VPS or a home lab, a service file is often the first building block — then you add Nginx as a reverse proxy in front of your app, set up Prometheus monitoring to watch for failures, and harden your SSH configuration so nobody gets in who shouldn’t.
Each piece builds on the last. But it all starts with a service file — a 15-line text file that turns your script into something the operating system respects as a first-class citizen.
Quick Reference Card
Bookmark this. I still check these commands weekly after a decade of Linux administration:
# Create/edit a service
sudo vim /etc/systemd/system/myservice.service
# Reload, start, enable
sudo systemctl daemon-reload
sudo systemctl start myservice
sudo systemctl enable myservice
sudo systemctl enable --now myservice # start + enable in one
# Status and logs
systemctl status myservice
journalctl -u myservice -f # follow logs live
journalctl -u myservice --since today # today's logs only
# Management
systemctl stop myservice
systemctl restart myservice
systemctl disable myservice
systemctl is-enabled myservice # check if enabled
# List everything
systemctl list-units --type=service
systemctl list-timers # scheduled jobs summary
Write your first service file today. Pick something you’re currently running in a tmux pane or screen session. Give it a proper service file, enable it, and reboot your server to make sure it comes back. The confidence that comes from knowing your script will survive a reboot — without you babysitting it — is worth the 15 minutes it takes to learn.