Automating tasks in linux

Automate your Linux system tasks with cron

It might surprise the saltiest of system administrators to know that there is no Linux cron command. The cron daemon ( crond ) is a system-managed executable that runs in memory with which users may schedule tasks. The user command to work with the cron service is crontab (cron table). The crontab file is a simple text file that instructs the cron daemon to perform a task at a certain time or interval. Any user may schedule cron tasks or jobs on a system. The task runs under the user account from which it was created. In other words, if you create a cron task, it runs with your user account’s permissions. The same is true for any user on the system, including the root user.

Using cron

You can issue the man crontab command to see all possible options, but there are generally two that work for most users: -l (list) and -e (edit).

To see a list of your configured cron tasks, use:

I have no cron tasks running yet. To create one, I need to edit my crontab file:

Note: There is no file name or designation of any kind required when creating a crontab entry. Each user has only one crontab file and you add all tasks to it.

More Linux resources

There is nothing particularly special about the crontab editor except that, if you don’t have a specific editor defined with either the VISUAL or the EDITOR environment variable, then your editor is probably either vi or vim .

Note: If you don’t know how to use vi or vim , please refer to my article, An introduction to the vi editor.

Press the I key to enter INSERT mode and begin setting up your crontab entry. The crontab file has a specific syntax to it. See the image below:

Making it work

As you can see from the image, the positioning of your entries has meaning, and the entries are separated by spaces. An asterisk ( * ) means every or all, as in every minute or all hours, every day, and so on. In other words, the image currently illustrates a crontab entry for a script that will run every minute of every day. This practice, while rare, does exist in some instances. However, for most cron tasks, you will need to be more specific.

Note: Before setting up a cron task, run your script or command to see if you have any problems with permissions or paths.

It’s a good practice to provide the full path to any executable inside a script. Likewise, you should also provide the full path to executables or scripts in your crontab entries.

For example, if you have a script that needs to create a file from the dmesg command once a day at 1pm, your entry looks like the following:

And if you want to email this file to yourself, you can add an additional entry into your crontab :

Save the file by exiting it as you would exit vi or your default editor. There is nothing special you have to do to enable this edited crontab . The cron daemon checks for the existence of entries in the /var/spool/cron directory automatically. For example, my crontab is /var/spool/cron/khess and contains the two entries above.

The cron daemon runs as root so it can read the directory’s contents since only the root user has access to this directory. It would be a serious security violation for other users to access /var/spool/cron and read the crontab entries for other users on the system.

More examples

For the following examples, assume that you have a script, backup.sh , that you want to schedule on your system. You’ve placed all your system scripts in /etc/scripts , which is a directory that only root has access to.

To schedule the backup script to run every night at 2am, open the root user’s crontab :

then enter the following:

Save and exit the file.

Let’s try something a little more advanced. Schedule the backup script to run at 2am every Monday. Now, what does the crontab entry look like? Refer back to the crontab syntax image for hints:

The 1 in the fifth column instructs cron to run this script on Monday.

Rather than have your special backup script run every Monday, run it on the 15th of every month:

What if the backup script only backs up a single mission-critical directory? You want that directory’s contents backed up every 15 minutes. Here’s how that would look, which isn’t 100% intuitive:

The traditional method of setting this every 15-minute schedule looks like the following:

And similarly, if you want to run the script only Monday through Friday, your entry changes to:

As you can see, cron scheduling is easy once you understand the syntax. The only real stumbling blocks you might experience with cron are pathing, permissions, and timing. You have to think about how long a script requires to execute and produce output before scheduling another process that depends on it. Referring back to the dmesg script, you can see that you need to know how long the dmesg command requires to finish processing and writing to the dmesg.txt file before scheduling the next command, which is to email the dmesg.txt file to yourself.

Wrapping up

Typically, the way system administrators and other users automate processes on a Linux system is to create scripts that perform functions such as creating files, moving files, emailing information, performing backups, reporting on backups, etc. When you schedule those same scripts in cron, you’ve created true automation on your system. There are very few repetitive tasks that cannot be automated using cron and scripts. If you experience problems with automating tasks that require passwords or interactive sessions, use expect scripts to automate those. If you don’t know expect , watch Enable Sysadmin for future posts on the topic.

There are other ways to automate things in Linux in addition to cron . For example, the at command can be used to run a job at a specific time. You can also set tasks to run at specific times by using systemd, though the systemd timers system.

[Want to try out Red Hat Enterprise Linux? Download it now for free.]

Источник

Automating tasks in linux

In Linux, tasks can be configured to run automatically within a specified period of time, on a specified date, or when the system load average is below a specified number. Red Hat Enterprise Linux is pre-configured to run important system tasks to keep the system updated. For example, the slocate database used by the locate command is updated daily. A system administrator can use automated tasks to perform periodic backups, monitor the system, run custom scripts, and more.

Red Hat Enterprise Linux comes with several automated tasks utilities: cron , at , and batch .

Cron is a daemon that can be used to schedule the execution of recurring tasks according to a combination of the time, day of the month, month, day of the week, and week.

Cron assumes that the system is on continuously. If the system is not on when a task is scheduled, it is not executed. To schedule one-time tasks, refer to Section 27.2 At and Batch .

To use the cron service, the vixie-cron RPM package must be installed and the crond service must be running. To determine if the package is installed, use the rpm -q vixie-cron command. To determine if the service is running, use the command /sbin/service crond status .

The main configuration file for cron, /etc/crontab , contains the following lines:

SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root HOME=/ # run-parts 01 * * * * root run-parts /etc/cron.hourly 02 4 * * * root run-parts /etc/cron.daily 22 4 * * 0 root run-parts /etc/cron.weekly 42 4 1 * * root run-parts /etc/cron.monthly

The first four lines are variables used to configure the environment in which the cron tasks are run. The SHELL variable tells the system which shell environment to use (in this example the bash shell), while the PATH variable defines the path used to execute commands. The output of the cron tasks are emailed to the username defined with the MAILTO variable. If the MAILTO variable is defined as an empty string ( MAILTO=»» ), email is not sent. The HOME variable can be used to set the home directory to use when executing commands or scripts.

Each line in the /etc/crontab file represents a task and has the following format:

minute hour day month dayofweek command

minute — any integer from 0 to 59

hour — any integer from 0 to 23

day — any integer from 1 to 31 (must be a valid day if a month is specified)

month — any integer from 1 to 12 (or the short name of the month such as jan or feb)

dayofweek — any integer from 0 to 7, where 0 or 7 represents Sunday (or the short name of the week such as sun or mon)

command — the command to execute (the command can either be a command such as ls /proc >> /tmp/proc or the command to execute a custom script)

For any of the above values, an asterisk (*) can be used to specify all valid values. For example, an asterisk for the month value means execute the command every month within the constraints of the other values.

A hyphen (-) between integers specifies a range of integers. For example, 1-4 means the integers 1, 2, 3, and 4.

A list of values separated by commas (,) specifies a list. For example, 3, 4, 6, 8 indicates those four specific integers.

The forward slash (/) can be used to specify step values. The value of an integer can be skipped within a range by following the range with / integer > . For example, 0-59/2 can be used to define every other minute in the minute field. Step values can also be used with an asterisk. For instance, the value */3 can be used in the month field to run the task every third month.

Any lines that begin with a hash mark (#) are comments and are not processed.

As shown in the /etc/crontab file, the run-parts script executes the scripts in the /etc/cron.hourly/ , /etc/cron.daily/ , /etc/cron.weekly/ , and /etc/cron.monthly/ directories on an hourly, daily, weekly, or monthly basis respectively. The files in these directories should be shell scripts.

If a cron task is required to be executed on a schedule other than hourly, daily, weekly, or monthly, it can be added to the /etc/cron.d/ directory. All files in this directory use the same syntax as /etc/crontab . Refer to Example 27-1 for examples.

# record the memory usage of the system every monday # at 3:30AM in the file /tmp/meminfo 30 3 * * mon cat /proc/meminfo >> /tmp/meminfo # run custom script the first day of every month at 4:10AM 10 4 1 * * /root/scripts/backup.sh

Example 27-1. Crontab Examples

Users other than root can configure cron tasks by using the crontab utility. All user-defined crontabs are stored in the /var/spool/cron/ directory and are executed using the usernames of the users that created them. To create a crontab as a user, login as that user and type the command crontab -e to edit the user’s crontab using the editor specified by the VISUAL or EDITOR environment variable. The file uses the same format as /etc/crontab . When the changes to the crontab are saved, the crontab is stored according to username and written to the file /var/spool/cron/ username .

The cron daemon checks the /etc/crontab file, the /etc/cron.d/ directory, and the /var/spool/cron/ directory every minute for any changes. If any changes are found, they are loaded into memory. Thus, the daemon does not need to be restarted if a crontab file is changed.

The /etc/cron.allow and /etc/cron.deny files are used to restrict access to cron. The format of both access control files is one username on each line. Whitespace is not permitted in either file. The cron daemon ( crond ) does not have to be restarted if the access control files are modified. The access control files are read each time a user tries to add or delete a cron task.

The root user can always use cron, regardless of the usernames listed in the access control files.

If the file cron.allow exists, only users listed in it are allowed to use cron, and the cron.deny file is ignored.

If cron.allow does not exist, users listed in cron.deny are not allowed to use cron.

To start the cron service, use the command /sbin/service crond start . To stop the service, use the command /sbin/service crond stop . It is recommended that you start the service at boot time. Refer to Chapter 12 Controlling Access to Services for details on starting the cron service automatically at boot time.

Источник

Top 10 Tools to Automate Linux Admin Tasks

If you are a Linux administrator, or you want to become one, there are certain tasks that can become repetitive and boring. In fact, back in the day, some tasks were so incredibly repetitive that it became very hard to keep track of all the servers; this is why automation tools were created to help with such tasks. These tools help you manage and administer different servers or systems at once, and some of them allow you to do a particular task with only a click or a command line. These tools mentioned below will definitely help you automate some of the tasks of a system administrator so that you can concentrate on other interesting concepts and tasks.

Here are the top 10 Linux Admins to boost office productivity and ease of access. Click on the links to visit the homepage.

1. Puppet

Puppet is one of the largest open-source system administration tools, which helps you automate several tasks. It is used by major enterprises to configure and administer their servers. It monitors the network continuously for any alterations, and it can automatically configure any changes that may happen. It also helps to create and manage users efficiently. It also allows different system administrators and developers to work together, which makes the tasks of each individual easier.

Puppet automates server set-up, program installation as well as system management. It is not only supported on Linux but also on Windows and macOS. It is not very hard to set up or use. It is a paid software, but since so many companies use it, it is worth the price.

2. CFEngine

CFEngine is an automation tool, which helps you with managing your infrastructure easily. It can search for system errors efficiently and fix them automatically, without the need for any user interaction. It can also automatically perform package updates across hundreds of servers at once. It can also help you do several tasks such as process management, schedule management, reporting and compliance, service management, password management, application deployment, etc. It is well known due to its ability to self-heal and its quick response times. It is relatively easy to download and set up and also easy to use once installed. It is not free or open-source, but since it is so well known, organizations are willing to buy it for efficiency.

3. Nagios

Nagios, also known as Nagios Core, is a management tool made for Linux system administration tasks that helps you automate and manage all systems in your infrastructure. It provides a thorough report on network traffic, which helps in monitoring any system threats that might damage the health of the network. It can help you manage printers, routers, etc. If it finds a fault in a system, it automatically restarts it to fix it. It is not free, but many enterprises like to use it. It is not very easy to work with, as it has a very complicated installation process, but once it is set up, it is very efficient. You can download it directly from their website and start working with it.

4. Cacti

Cacti is an open-source, web-based network monitoring tool for Linux. It works in real-time, which makes it very efficient at finding any discrepancy in the system quickly. It collects data about the different systems under it and presents it in the form of an easy to understand graph format. It collects data from several different data centers and stores them in a MySQL database, which makes it easier for administers to monitor the server load, network performance, and such tasks.

It also has a user management tool, which you can use to give access to certain areas of the tools to users that you can add. Cacti is quite hard to install and set up, it requires some expert help, but it is easy to use once it has been installed. It is a paid service, and it is not the easiest to use.

5. Amanda

Amanda is one of the oldest Linux backup tools on the market. It is open-source, and it is widely popular. Using this tool, you can automate, schedule, and track backups. It can backup multiple hosts on your network at the same time. The backups can be stored anywhere, from disks, tapes, or optical media. It is not the easiest to set-up, but once it is running, it consistently backs up data to ensure that nothing is lost. It is normally free, but it has an enterprise version, which is not. The enterprise version gives you some extra tools such as Centralized backup of heterogeneous systems, support for a large collection of OS, etc.

6. Bacula

Bacula is a collective set of programs that can manage the backup, recovery, and verification of computer data from different networks. It can help you backup different types of media as data. It can work from a single computer and manage hundreds of computers at once. There is a very user-friendly admin workstation, where you can either use command lines or a GUI to interact with the software. It also consists of a backup server, which is where the background application will run to automate and schedule the data backup.

Bacula is not a very easy tool to use; it has a complicated system to set up and does not have as many features as the previous one. However, it is simple and easy to use once it is set up. It has a free version; however, the enterprise version is paid.

7. SaltStack

Salt or SaltStack is a Python-based open-source software, which is used for IT automation and remote task execution. It ensures high-speed data collection and system management for massive infrastructure. Salt is primarily used for remote tasks, which can be as simple as asking a web server to show a webpage and as complex as using a shell session to issue commands.

Salt is generally used by massive enterprises because it is efficient at managing all servers at once. With a single command line or a click of a button, you could update the software on all servers. It has high security as it uses data encryption to protect all data that is processed through it. It has a relatively easy installation process, and also it is easy to use. It is not free, but the price is worth it as it is very efficient.

8. Cockpit

The Cockpit is an open-source server manager, which is very easy to use. It is developed by Red Hat, one of the leading open-source software producers. The cockpit is a web-based GUI, which helps you automate several Linux administrative tasks. It helps you administer several servers at the same time.

Cockpit allows you to automate system software updates and security updates. It also allows you to do tasks such as software installations and user management. You can install Cockpit on most Linux distros, but it is pre-installed on CentOS.

9. cPanel

It is the most reliable web-based administrative tool which lets you manage several things such as websites, domains, apps, etc. One of the key features of cPanel is its integration with WHM for the automation of daily tasks using hooks. The hook system helps you to automatically create various items in your servers, such as user accounts, emails, logs, MySQL databases, and more.

The integration with WHMCS allows for automatic billing management. It has a lot of features, and you can use it with a powerful GUI to manage all kinds of hosting services and more. It is quite easy to install and use. It is not free, but it is very useful, so it is worth the money.

10. phpMyAdmin

phpMyAdmin is a PHP-based open-source web app. It is free to use, and you can use it on a web browser for creating and managing MySQL databases. It is preferred by students and junior system admins because of it’s easy to use interface. It is used for several administrative tasks, such as automating MySQL databases, along with creating, deleting, and altering them. It can work with different operating systems as well. It can run MySQL queries, optimize, repair, and check tables and do other tasks related to database management. It is easy to install, and since it is free, it is available to everyone for use.

So, these are the top 10 tools to automate Linux admins tasks. Feel free to share with us your view about the above tools at @linuxhint and @SwapTirthakar.

About the author

Swapnil Tirthakar

A Software Engineer who loves football and passionate about traveling. I often spend my free time playing with gadgets and exploring new possibilities in tech world. I am Linux enthusiast and have about 6 years of experience in web development. I have good command on Python, Java, SQL and system security.

Источник