Automated Backup in Linux using Shell Scripting and Crontab Scheduler

Reading time: 40 minutes | Coding time: 10 minutes

In today's world, data is very valuable. Every action on the computing devices generates data from which value can be derived. For example, large E-commerce companies like Amazon track our activity on their website and generate advertisements and recommendation according to our previous choices. Since data and information is valuable, we must have a secure backup of it for easy recovery in case of data loss.

Our main objective in this article is to take backup of a specified folder in Linux by compressing it and storing it in a backup directory.
For Compression we use the TAR (Tape Archive) tool.
For Scheduling the backup process, we use Crontab Scheduler.

We will cover the following sub-topics:

1. Shell Script for backup of a defined folder
2. Shell Script for backup of multiple folder
3. Shell Script to take backup with date filter
4. Shell Script to automatically delete backup after certain days
5. Scheduling the scripts using Crontab Scheduler

Creating a backup folder

Create a folder where backup will be taken.

Generally, it is preferred to take backup in remote repositories.
Here, the backup folder is created on the Root of the Linux Machine in sudo priviliges which provides security but it can be easily deployed in remote repository.

For creating the backup folder:

1. Switch to the root user

$ sudo su

The Shell prompt changes from $ to # for root user.

2. Go to the Root folder

# cd /

3. Create Backup folder

# mkdir backupfolder 

By using the ls command to list the contents of the specified folder, it can be seen the backupfolder/ is created at the root directory.

1. Shell Script for backup of defined folder

A shell script is to be written for following tasks:

1. Create a compressed copy of the folder of which backup is to be taken using TAR.
2. Move the compressed copy to the backup folder.

Creating a Shell Script file

# touch <filename.sh>

Creates a file named filename.sh. Here, file name is backup.sh

Provide executable permission to the Shell Script

# chmod +x <filename.sh>

Verify the creation of Shell Script

# ls

The backup.sh file is created and it is highlighted in green color meaning it has executable permissions.

Code:

#!/bin/bash

#tar = Tape Archive tool for compression
tar -cvf /backupfolder/backup.tar /home/nishkarshraj/Desktop/Automation-using-Shell-Scripts

#Go to the backup folder location
cd /backupfolder

#Show the size of the folder
du -sh

Explanation of the code:

1. Tar implementation:

Syntax:

tar -cvf /{absolute path to destination folder}/filename.tar /{absolute path of files to be compressed}/

• Flag -c is a mandatory flag used to specify Creation of tar file
• Flag -v is an optional flag used to print Verbose of the tar file creation, i.e., it prints all the file compression progress to the Shell Command line.
• Flag -f is a mandatory flag used to access all the files in the specified path.
• Absolute path means the path relative to root folder /
• filename.tar is the name of the file to be created after compression with a extension .tar
• /backupfolder/filename.tar specifies that the filename.tar is created and moved to /backupfolder.

2. Specifying the Disk Usage

• du is the command to check disk usage of the current folder in Linux
• -s tag is used to print in summary form
• -h tag is used to print the output in human readable form.

Execute the Shell Script

# ./backup.sh

Here, the last portion of output is shown.

• All files progress in compression is printed to standard output because -v tag is used in Tar.
• At last, 3.4 MB disk usage is printed.

See the contents of backupfolder to check is the backup is successful

# ls backupfolder/

• backup.tar is successfully stored in backupfolder/

2. Shell Script to take backup of multiple folders in one compressed file

The first task of taking backup of one folder is ideal.
Generally, we store our valuable data and information in organized manner in different folders, thus backup for all important folders must be taken.

This can be handled easily because tar allows multiple files as input for creation of a single compressed file.

Syntax:

tar cvf /{destination path}/filename.tar {path to folder 1} {path to folder 2} ...

Thus, all folders specified will be compressed and transferred to the destination folder of .tar file.

Code:

#!/bin/bash

#Specify folders whose backup is to be taken in variables
f1="/home/nishkarshraj/Desktop/Automation-using-Shell-Scripts"
f2="/home/nishkarshraj/Desktop/Computer-Graphics"
f3="/home/nishkarshraj/Desktop/HelloWorld"

#tar = Tape Archive tool for compression
#Creating same backup tar file for all specified folders
tar -cvf /backupfolder/backup.tar $f1 $f2 $f3

#Go to the backup folder location
cd /backupfolder

#Show the size of the folder
du -sh

Here, we statically define the folders in variables whose backup is to be taken.
All the folders are passed as argument to tar command using the $ symbol to fetch their values.

Code in vi-editor:

Output:

Here, the final part of the output of execution of shell script is shown.
It can be easily seen that it includes:

1. files from Computer-Graphics folder
2. file from HelloWorld folder
3. Disk usage of backup folder: 4 MB

3. Shell Script to take backup with dates

Till now, a common name backup.tar is used for backup of files in /backupfolder.
The biggest disadvantage of above is that the daily progress cannot be monitored and it overwrites the previous day backup.tar on creation.

This problem can be tackled by creating backup files with their day of creation concatenated at the end of their name.

Example: backup_15_09_2019.tar for backup taken on 15th September, 2019.

Code:

#!/bin/bash

# Compress the folder with foldername + date and take backup
filename="backup_`date +%d`_`date +%m`_`date +%Y`.tar";

# Create compressed file using tar and move to backup folder
tar cvf /backupfolder/$filename /home/nishkarshraj/Desktop/HelloWorld

# Move to Backup folder
cd /backupfolder

# List the content
ls

# List the disk usage
du -sh

Here, date is a bash keyword storing date with parameters day, date, month, time, timezone and year.

• date +%d defines current date.
• date +%m defines current month.
• date +%Y defines current year.

Code in vi-editor:

Output:

The following points can be noted from the output:

1. Shell takes backup of specified folder HelloWorld
2. Shell creates backup file with date at the end: backup_15_09_2019.tar
3. The size of backup folder is displayed: 16 KB

4. Shell Script to automatically delete backup after certain days

Taking backup daily improves the security of the data and makes it easy to recover in case of data loss but taking backup is an overhead which requires lot of space and computing resources like space are limited and expensive.
Thus, it is ideal to delete backups after certain days because latest backups are more important and updated.

Here, we create a shell script to delete the backup.tar files that are older than five days.

Code:

#!/bin/bash

find "/backupfolder" -type f -mtime +5 -exec rm {} \;

• find command searches the specified directories and performs the command specified after it.
• -type f searches for files in the specified folder
• -mtime tells Linux to search files with timestamp in terms of date. Here, +5 indicates search the files older than 5 days.
• exec command is executed if files match above criteria.
• rm command is used to delete the matching files
• {} ; fetches all the matching files of find command and serves input to rm command one by one.

Code in vi-editor:

5. Scheduling Shell Scripts using Crontab Scheduler

Crontab Scheduler is inbuilt tool in Linux that automatically executes defined task at specified schedule.

Here, Crontab Scheduler is used to automatically take backup of specified folder using backup.sh shell script every day at 12 O'Clock in the afternoon.

Note: Crontab Schedulers perform their task at specified schedule given the machine is on power at that time.

Open Crontab for root user using the following command:

# crontab -e

Where -e tag is used to open the crontab in edit mode.

Crontab scheduler gives three ways to modify itself. Choose any one.

The Crontab file opens up.

Go at last of the Crontab and specify the command to be executed in following syntax:

Syntax:

(Minute) (Hour) (Date) (Month) (Day of Week) (Command)

• Minute: Minute at which the command is to be executed. It takes value from 0 to 59.
• Hour: Hour at which the command is to be executed. It takes value from 0 to 23.
• Date: Date of the month (1-31)
• Month: Month of the year (1-12)
• Day of the Week: It takes value from 0 to 6.
  0 : Sunday
  1 : Monday etc.
• Command: Command to be executed

Schedule specified:

0 12 * * * ./backup.sh 
0 12 * * * ./remove-backup.sh

The crontab executes the backup.sh file every day of the year at 12:00 pm and also finds and deletes the backup files that are older than five days.

Reference

Shell Scripting Source Code