Systemd, is a tool to manage processes and systems in linux. It offers a very powerful logging solution to log service related information through its journal system. The journalctl service provides a way to access and manipulate the data stored in the systemd journal.
The main goal of this systemd’s journal is to have a centralized log management, merge logs from multiple sources to a single and easy to access source. The Journald daemon is responsible for collecting and managing log messages from a variety of components such as the kernel, initrd and other services.
The main advantage of using systemd journal includes dynamic log displays, arbitrary output formats and the ability to replace or complement existing syslog implementation. This centralized approach gives an efficient log analysis, which enables the users to have log data presented according to one’s needs.
Setting the System Time
Before getting into journalctl, it is important to make sure the system time settings are correct. Systemd provides the timedatectl tool for managing time-related configs. Use it to list all available time zones and set the appropriate time zone for your servers.
sudo timedatectl set-timezone <zone>
Basic Log Viewing
To view logs stored by the journald daemon, utilize the journalctl command. When executed without options, it displays all journal entries in a pager. You can navigate through the logs, which include information from the early boot process, kernel, initrd, and application standard error and output.
To display timestamps in UTC instead of local time, use the –utc flag:
Journal Filtering by Time
Displaying Logs from the Current Boot
A common scenario is viewing logs from the current boot. The -b flag achieves this:
For previous boots, utilize the –list-boots option to identify available boots and display logs from a specific boot using the -b flag along with the boot ID.
journalctl -b <boot-ID>
To filter logs within specific time ranges, use the –since and –until options. Time values can be absolute or relative, allowing flexibility in defining time windows.
journalctl --since "YYYY-MM-DD HH:MM:SS"
journalctl --since yesterday
journalctl --since 09:00 --until "1 hour ago"
Filtering by Message Interest
Filtering by unit (service) is useful for focusing on specific components. Use the -u option to retrieve logs related to a particular unit.
journalctl -u nginx.service
journalctl -u nginx.service --since today
journalctl -u nginx.service -u php-fpm.service --since today
By Process, User, or Group ID
Filter logs by process ID (_PID), user ID (_UID), or group ID (_GID). This can be valuable for isolating logs associated with specific processes or users.
journalctl _UID=<user-ID> --since today
journalctl -F _GID # Display available group IDs
By Component Path
Filter logs by providing an executable path. This is helpful when focusing on entries related to a specific executable.
Displaying Kernel Messages
Retrieve kernel messages using the -k or –dmesg flags:
journalctl -k -b -5 # Display messages from five boots ago
Filter logs by priority using the -p option, showing entries at or above the specified level.
journalctl -p err -b
Modifying the Journal Display
Truncate or Expand Output
Adjust how journalctl displays data by using the –no-full option to truncate output or the -a flag to display all information.
Output to Standard Out
By default, journalctl uses a pager. Use the –no-pager option to output directly to standard output, facilitating further processing.
Change the output format using the -o option with format specifiers such as json, json-pretty, short, etc.
journalctl -b -u nginx -o json
journalctl -b -u nginx -o json-pretty
Active Process Monitoring
Journalctl can serve as a real-time log monitoring tool.
Displaying Recent Logs
Use the -n option to display a specific number of recent log entries.
journalctl -n 20
Actively follow logs as they are written using the -f flag, similar to tail -f.
Finding Current Disk Usage
Check the current disk usage of the journal with the –disk-usage flag:
Deleting Old Logs
Shrink the journal by specifying a size with –vacuum-size or a cutoff time with –vacuum-time.
sudo journalctl --vacuum-size=1G
sudo journalctl --vacuum-time=1years
Limiting Journal Expansion
Configure journal growth limits in the journald.conf file, using options like SystemMaxUse, SystemKeepFree, etc.
The Journalctl command, coupled with systemd’s journal, offers a more robut solution for log management and analysis. The flexibility it offers, coupled with various filtering and formatting options, empowers the admins to efficiently navigate and extract valuable insights from system and app logs. Understanding this abilities of Journalctl enhances ability to troubleshoot, monitor, and maintain a systemd-based system effectively.