- Meanwhile years ago i’ve wrote an intro to systemd. Recently i had close contact with it, i want to write down some practical informations with focus on Systemd services.*
Systemd unit files as records of initialization instructions for each service in declarative style are a great idea. The declarative approach has a huge advantage of replacing the traditionally used per-daemon startup shell scripts. It mitigates differences between distributions and also maintenance on the time scale. See my overview over Unit types.
A service unit describes how a service is managed. This will include how to start or stop, restart the service, under which circumstances it should be automatically started, and the dependency and ordering information for related software.The easiest way to determine the type of a unit is with its type suffix, which is appended to the end of the resource name.
Most interesting commands for
systemctl start | stop | restart | reload | status | reload-or-restart | is-active [name.service] systemctl list-units --type service --all
systemctl status [name.service]
is your friend. It shows basic status information and several loglines of
journald logs for the service.
If you are unsure whether the service has the functionality to reload its configuration, you can use the
reload-or-restart command. This will reload the configuration in place if available. Otherwise, it will restart the service so the new configuration is picked up.
Unit file locations
Let’s go beyond basic usage. The first thing we should be familiar with, is where unit files are located on the system and in which order they are loaded.
/usr/lib/systemd/system/- units provided by installed packages
/etc/systemd/system/- units installed by the system administrator
or list the state with:
systemd-analyze unit-paths # and systemd-analyze --user unit-paths
Unit file Locations: system mode
Maybe it needs some more words of explanation. When
systemd starts its run in system mode and check a particular set of (system-mode)locations. Here is an example of this cascade from my Fedora installation:
#systemd-analyze unit-paths /etc/systemd/system.control /run/systemd/system.control /run/systemd/transient /run/systemd/generator.early /etc/systemd/system /run/systemd/system /run/systemd/generator /usr/local/lib/systemd/system /usr/lib/systemd/system /run/systemd/generator.late
Therefore you may put unit files in
/etc/systemd/system/ and it will override something that is coming by default with the package into
/usr/lib/systemd/system or is autogenerated from legacy scripts.
However, that is only the beginning. If
systemd is started in user mode another set of locations is processed.
systemd in user mode instance (
systemd --user) is launched when the user logs in for the first time. We can also check out these locations with
systemd-analyze --user unit-paths. However, it’s out of scope a little bit. Check man page for more details.
Listing Service Units
We can list all service units present on the system with the status by
systemctl list-unit-files --type service
for loaded services:
systemctl list-units --type service
It might be usefull to list failed units as well
systemctl list-units --failed
Edit Unit Files
There are two approaches
Replacement can be done e.g with
systemctl edit --full my-unit.service
/etc/systemd/system/my-unit.service in your editor, copying the installed version (if it does not exist yet) from
/usr/lib/systemd/system/my-unit.service. Finally on save it automatically reloads the unit.
Alternatively, to create drop-in files for the unit file
/usr/lib/systemd/system/my-unit.service, create the directory
/etc/systemd/system/my-unit.service.d/ and place .conf files there to override or add new options. Systemd will parse and apply these files on top of the original unit by mergin directives.
The easiest way to do this is to run:
systemctl edit my-unit.service
This opens the file /etc/systemd/system/my-unit.service.d/override.conf in your text editor (creating it if necessary) and automatically reloads the unit when you are done editing.
However, you can always revert status to default by:
systemctl revert my-unit.service
and check the overall units delta by
systemd-delta, here an example from my machine:
# systemd-delta [REDIRECTED] /etc/systemd/system/dbus-org.freedesktop.timedate1.service → /usr/lib/systemd/system/dbus-org.freedesktop.timedate1.service [EQUIVALENT] /etc/systemd/system/default.target → /usr/lib/systemd/system/default.target [MASKED] /etc/systemd/system/systemd-timedated.service → /usr/lib/systemd/system/systemd-timedated.service [EXTENDED] /usr/lib/systemd/system/systemd-udev-trigger.service → /usr/lib/systemd/system/systemd-udev-trigger.service.d/systemd-udev-trigger-no-reload.conf [EXTENDED] /usr/lib/systemd/user/dbus.service → /usr/lib/systemd/user/dbus.service.d/flatpak.conf
at this point, we should understand where to find what, let’s take a look at some service units.
Service unit example
Here is an example of system service
sshd. With command:
systemctl cat sshd.service
on my Fedora 28 linux i get
# /usr/lib/systemd/system/sshd.service [Unit] Description=OpenSSH server daemon Documentation=man:sshd(8) man:sshd_config(5) After=network.target sshd-keygen.target Wants=sshd-keygen.target [Service] Type=notify EnvironmentFile=-/etc/crypto-policies/back-ends/opensshserver.config EnvironmentFile=-/etc/sysconfig/sshd ExecStart=/usr/sbin/sshd -D $OPTIONS $CRYPTO_POLICY ExecReload=/bin/kill -HUP $MAINPID KillMode=process Restart=on-failure RestartSec=42s [Install] WantedBy=multi-user.target
[Unit] section is more or less self-explanatory. But important directives are
Wants. These two define dependencies on other units. Means
sshd should run after
sshd-keygen targets. Also,
sshd-keygen should be started, however,
sshd can start also if the start of
sshd-keygen has failed (Check Wants vs. Required) Of course a lot more options existing in this section.
[Service] section controls the behavior of the service. Here we already see important parts. E.g.
ExecStart defines which command starts the service.
ExecStart=/usr/sbin/sshd -D $OPTIONS $CRYPTO_POLICY
that command reload the configuration
ExecReload=/bin/kill -HUP $MAINPID
systemd waits until
sshd notifies it about successful start over Notify service manager, while
KillMode=process means that only the main process should be killed, a remaining process in the same control-group continuing to live. If not set this property defaults to control-group, which means everything in the same control-group is killed.