journalctl in Linux

In Linux, systemd provides a centrally managed journal where all kernel and userland processes' logs are stored. We use journalctl to interact with journals.

Table of contents.

1. Introduction.
2. Syntax.
3. Commands.
4. Summary.
5. References.

Introduction.

In Linux, system logs are stored in the /var/log directory:

ls /var/log

systemd provides a centrally managed journal where all kernel and userland processes' logs will be stored.

This journal contains logs collected from different services as can be seen from the output above and therefore instead of using different tools to collect different logs, we use a single utility to interact with all system logs.

The journal is implemented using journal daemon that collects data from all sources such as kernel, initrd, or other services and stores them in a binary format that can be dynamically manipulated.

We use the journalctl utility to access and manipulate data that is stored within these journals.

Syntax.

The syntax is as follows:

$ journalctl [OPTIONS...] [MATCHES...]

Commands.

In this section, we discuss commonly used journalctl commands.

1. View logs.

To view logs collected by journald we use the journalctl command as follows:

$ journalctl

2. Navigating the logs.

As we have seen, the output of journalctl is similar to the output of the less command. This is because the former pipes its output to the latter.

The following are ways which we use to navigate the logs.

• To move down by a single line we use the down arrow key or enter key or e or j

• To move up, we use the up arrow key or y or k.

• We use the space bar to move down by one page.

• To move up by one page we use the b key.

• The right and left arrow keys are used to move right and left respectively.

• We use the g and G to move the first and last lines respectively.

• To search for logs, we write /search term after which we press enter and all logs matching the search term will be located.
  

• We can also search backward from where we are by writing ?search term.

• During a search we use the n and N keys to move to the next and previous occurrences respectively.

• To quit we press the q option.
  Just to name a few.

3. Reverse order.

The above command prints out logs starting from the oldest to the latest. We can opt to print logs starting from the latest. In this case, we use the -r option as follows:

$ journalctl -r

4. Journatclt output.

We can format journalctl output in various formats such as JSON, verbose, short precise, short, etc.

For example to print system logs formatted in verbose we write:

$ journalctl -o verbose

To print output in JSON we write:

$ journalctl -o json

5. Log entries.

We can also choose the number of logs we need to be displayed by using the -n option followed by the number of logs:

$ journalctl -n 3

The above command only prints out the latest 10 logs.

6. Priority logs.

We can print out logs with a specified priority, that is, emerg which is (0),
alert = (1), crit = (2), err = (3), warning = (4), notice = (5), info = (6), debug = (7).
For example to print logs that are critical we write:

$ journalctl -n 3 -p err

We can also use the priority number:

$ journalctl -p 2

Kernel logs.

The -k option is used to display logs from the kernel.

$ journalctl -n 3 -k

Searching for logs.

We can pipe the output of the journalctl command to a text processing utility such as grep to search for specific logs.
For example to get system logs involving a specific hostname we write:

$ journalctl | grep [hostname]

System boots.

System logs also include when the system was booted. We can view boots by using the --list-boots option:

$ journalctl --list-boots

After we have listed boots, we can get logs for a specific boot by specifying it as follows:

$ journalctl -b [boot number or ID]

Service logs.

To list logs involving a specific service unit we use the -u option. For example to display logs involving the ssh service unit we write:

$ journalctl -u ssh

We can also provide a path to an executable as follows:

$ journalctl -u /usr/bin/ssh

UID, PID, GID logs.

We can get logs for a specific user by specifying their user ID (UID) as follows:

$ journalctl _UID=[user ID]

We can also get logs using the Process ID as follows:

$ journalctl _PID=[Process ID]

To get logs for a group, we use the group ID as follows:

$ journalctl _GID=[Group ID]

Real-time logs.

We use the -f option to print recent logs. This option will also continuously print new journal entries as they are appended to the journal.

$ journalctl -f

Time specific logs.

Sometimes we want logs from a specific period, for this we use the -S or --since option that shows logs that are not older than the date we specify and the -U or --until option which displays logs not newer than the date we specify.
For example, to view logs from yesterday, we write:

$ journalctl --since yesterday

This is referred to as a relative time, we could also write 1d signifying one day in the past, 1h for the last hour, today, etc.

To get logs for a specific date we write:

$ journalctl --since YYYY-MM-DD HH:MM:SS

We can also specify a time range as follows:

$ journalctl --since "YYYY-MM-DD HH:MM:SS" --until "YYYY-MM-DD HH:MM:SS"

We can also omit the time and just specify the date as follows:

$ journalctl --since YYYY-MM-DD

Quiet mode.

Notice from the output that we display irrelevant text such as warnings, we can choose to ignore such by using the --quiet option as follows:

$ journalctl --quiet

Archived logs.

All these logs are stored on the disk. To manage disk space we may need to delete logs when they increase beyond a certain threshold.
To view disk usage by the journal we write:

$ journalctl --disk-usage

We can delete logs until we are left with logs of size 2GB by writing:

$ journalctl --vacuum-size=2G

We can also delete logs that are older than a specified time. For example to remove logs older than 1 year we write:

$ journalctl --vacuum-time=1years

Summary.

Journal logs are stored in binary format and are accessed anytime using the journalctl command.

We use the journalctl command to manage and manipulate all recorded logs.
We can combine multiple journalctl command options to obtain the desired output.

References.

For journalctl manual we write $ man journalctl.