Checksum in linux on file

Learn How to Generate and Verify Files with MD5 Checksum in Linux

A checksum is a digit which serves as a sum of correct digits in data, which can be used later to detect errors in the data during storage or transmission. MD5 (Message Digest 5) sums can be used as a checksum to verify files or strings in a Linux file system.

MD5 Sums are 128-bit character strings (numerals and letters) resulting from running the MD5 algorithm against a specific file. The MD5 algorithm is a popular hash function that generates 128-bit message digest referred to as a hash value, and when you generate one for a particular file, it is precisely unchanged on any machine no matter the number of times it is generated.

It is normally very difficult to find two distinct files that results in same strings. Therefore, you can use md5sum to check digital data integrity by determining that a file or ISO you downloaded is a bit-for-bit copy of the remote file or ISO.

In Linux, the md5sum program computes and checks MD5 hash values of a file. It is a constituent of GNU Core Utilities package, therefore comes pre-installed on most, if not all Linux distributions.

Take a look at the contents of /etc/group saved as groups.cvs below.

The md5sums command below will generate a hash value for the file as follows:

When you attempt to alter the contents of the file by removing the first line, root:x:0: and then run the command for a second time, try to observe the hash value:

You will notice that the hash value has now changed, indicating that the contents of the file where altered.

Now, put back the first line of the file, root:x:0: and rename it to group_file.txt and run the command below to generate its hash value again:

From the output above, the hash value is still the same even when the file has been renamed, with its original content.

Important: md5 sums only verifies/works with the file content rather than the file name.

The file groups_list.txt is a duplicate of groups.csv, so, try to generate the hash value of the files at the same time as follows.

You will see that they both have equal hash values, this is because they have the exact same content.

You can redirect the hash value(s) of a file(s) into a text file and store, share them with others. For the two files above, you can issues the command below to redirect generated hash values into a text file for later use:

To check that the files have not been modified since you created the checksum, run the next command. You should be able to view the name of each file along with “OK”.

The -c or —check option tells md5sums command to read MD5 sums from the files and check them.

Remember that after creating the checksum, you can not rename the files or else you get a “No such file or directory” error, when you try to verify the files with new names.

The concept also works for strings alike, in the commands below, -n means do not output the trailing newline:

In this guide, I showed you how to generate hash values for files, create a checksum for later verification of file integrity in Linux. Although security vulnerabilities in the MD5 algorithm have been detected, MD5 hashes still remains useful especially if you trust the party that creates them.

Verifying files is therefore an important aspect of file handling on your systems to avoid downloading, storing or sharing corrupted files. Last but not least, as usual reach us by means of the comment form below to seek any assistance, you can as well make some important suggestions to improve this post.

If You Appreciate What We Do Here On TecMint, You Should Consider:

TecMint is the fastest growing and most trusted community site for any kind of Linux Articles, Guides and Books on the web. Millions of people visit TecMint! to search or browse the thousands of published articles available FREELY to all.

If you like what you are reading, please consider buying us a coffee ( or 2 ) as a token of appreciation.

We are thankful for your never ending support.

Источник

How to generate and verify file checksums on Linux

This article describes how to generate MD5 and SHA file checksums and verify file integrity on Linux.

What is a checksum?

A checksum (also sometimes referred to as a hash) is an alphanumeric value that uniquely represents the contents of a file. Checksums are often used to verify the integrity of files downloaded from an external source, such as an installation file. You can also use checksums to verify the integrity of your own files. For example, you can generate checksums for your backup files and then use checksum algorithms or a hashing utility to ensure they have not become corrupted or altered at a later date. If both checksums are the same, the files are the same.

The two most commonly used checksum algorithms are MD5 and SHA. When verifying checksums, you must make sure you use the same algorithm that was used to generate the checksum. For example, a file’s MD5 checksum value is completely different from its SHA-256 checksum value.

“SHA” is an umbrella term for an entire family of hashing algorithms. The two most commonly used SHA algorithms are:

• SHA-1
• SHA-2: This family includes several hashing functions, whose names indicate the hash value bit-length (for example, SHA-224, SHA-256, SHA-384, and SHA-512).

Generating checksums on Linux

All A2 Hosting servers run Linux, and therefore include command line programs for generating MD5 and SHA checksums.

To generate a file checksum and store the value in a file, follow these steps:

1. Log in to your account using SSH.
2. At the command prompt, type one of the following commands, replacing filename with the name of the file for which you want to generate a checksum:
   • To generate an MD5 checksum, type:

To generate an SHA checksum, type the name of the command for the hashing algorithm you want to use. For example, to generate a SHA-256 checksum, use the sha256sum command. To generate a SHA-512 checksum, you would type the following command:

The md5sums.txt (or sha512sums.txt) file now contains a file listing and associated checksums.

You can also generate multiple checksums at once. For example, to generate MD5 checksums for all of the .zip files in the current directory, type the following command:

Similarly, to generate MD5 checksums for all of the files in the current directory and all directories beneath it, type the following command:

To generate SHA checksums instead, replace md5sum with the appropriate SHA command.

Verifying checksums on Linux

All A2 Hosting servers run Linux, and therefore include command line programs for verifying MD5 and SHA checksums.

To verify file checksums, follow these steps:

1. Log in to your account using SSH.
2. At the command prompt, type one of the following commands based on the algorithm (MD5 or SHA) that was used to generate the checksums. These examples use the filenames md5sums.txt and sha512sums.txt. If you stored the checksums in a different file, use that filename instead.
   • To verify MD5 checksums, type:

To verify SHA checksums, type the name of the command for the hashing algorithm you want to use. For example, to verify a SHA-256 checksum, use the sha256sum command. To verify a SHA-512 checksum, you would type the following command:

Each matching checksum displays OK, while a mismatched checksum displays FAILED.

If you are checking a lot of files, mismatched results can get lost amongst all of the scrolling results. To display only mismatched checksums, use the —quiet option. For example:

Other operating systems

You can generate and verify checksums on other operating systems:

• For a computer running Microsoft Windows, you can install one of the many programs available for download that generates and verifies checksums.
• For a computer running Apple macOS, you can use the md5 and shasum programs at the command line.

Источник

An introduction to hashing and checksums in Linux

Photo by Markus Spiske from Pexels

The chances are that you’ve seen references to hashes or checksums when you’ve downloaded software from the Internet. Often, the software will be displayed, and then near the link is a checksum. The checksum may be labeled as MD5, SHA, or with some other similar name. Here is an example using one of my favorite old games from the 1990s named Nethack:

Many people don’t know exactly what this information means or how to work with it. In this article, I discuss the purpose of hashing, along with how to use it.

Goals of cryptography

In this first section, I want you to unlearn something. Specifically, I want you to break the association in your head between the word encryption and the word confidential. Many of us conceive of these two words as being synonymous when that is actually not the case. Cryptography, which includes encryption, can provide confidentiality, but it can also satisfy other goals.

Cryptography actually has three goals:

• Confidentiality — to keep the file content from being read by unauthorized users
• Authenticity — to prove where a file originated
• Integrity — to prove that a file has not changed unexpectedly

Linux security

It is that third concept, integrity, that we are interested in here. In this context, integrity means to prove that data has not changed unexpectedly. Proving integrity is useful in many scenarios:

• Internet downloads such as Linux distributions, software, or data files
• Network file transfers via NFS, SSH, or other protocols
• Verifying software installations
• Comparing a stored value, such as a password, with a value entered by a user
• Backups that compare two files to see whether they’ve changed

What is hashing?

Cryptography uses hashing to confirm that a file is unchanged. The simple explanation is that the same hashing method is used on a file at each end of an Internet download. The file is hashed on the web server by the web administrator, and the hash result is published. A user downloads the file and applies the same hash method. The hash results, or checksums, are compared. If the checksum of the downloaded file is the same as that of the original file, then the two files are identical, and there have been no unexpected changes due to file corruption, man-in-the-middle attacks, etc.

Hashing is a one-way process. The hashed result cannot be reversed to expose the original data. The checksum is a string of output that is a set size. Technically, that means that hashing is not encryption because encryption is intended to be reversed (decrypted).

What kind of hash cryptography might you use with Linux?

Message Digest and Secure Hash Algorithm

In Linux, you’re likely to interact with one of two hashing methods:

These cryptography tools are built into most Linux distributions, as well as macOS. Windows does not typically include these utilities, so you must download them separately from third party vendors if you wish to use this security technique. I think it’s great that security tools such as these are part of Linux and macOS.

Message Digest versus Secure Hash Algorithm

What’s the difference between the message digest and secure hash algorithms? The difference is in the mathematics involved, but the two accomplish similar goals. Sysadmins might prefer one over the other, but for most purposes, they function similarly. They are not, however, interchangeable. A hash generated with MD5 on one end of the connection will not be useful if SHA256 is used on the other end. The same hash method must be used on both sides.

SHA256 generates a bigger hash, and may take more time and computing power to complete. It is considered to be a more secure approach. MD5 is probably good enough for most basic integrity checks, such as file downloads.

Where do you find hashing in Linux?

Linux uses hashes in many places and situations. Checksums can be generated manually by the user. You’ll see exactly how to do that later in the article. In addition, hash capabilities are included with /etc/shadow , rsync , and other utilities.

For example, the passwords stored in the /etc/shadow file are actually hashes. When you sign in to a Linux system, the authentication process compares the stored hash value against a hashed version of the password you typed in. If the two checksums are identical, then the original password and what you typed in are identical. In other words, you entered the correct password. This is determined, however, without ever actually decrypting the stored password on your system. Check the first two characters of the second field for your user account in /etc/shadow . If the two characters are $1, your password is encrypted with MD5. If the characters are $5, your password is encrypted with SHA256. If the value is $6, SHA512 is being used. SHA512 is used on my Fedora 33 virtual machine, as seen below:

How to manually generate checksums

Using the hash utilities is very simple. I will walk you through a very easy scenario to accomplish on a lab computer or whatever Linux system you have available. The purpose of this scenario is to determine whether a file has changed.

First, open your favorite text editor and create a file named original.txt with a line of text that reads: Original information.

Next, run the file through a hash algorithm. I’ll use MD5 for now. The command is md5sum . Here is an example:

Kubernetes and OpenShift

Notice the resulting checksum value. This value is large enough that it’s difficult to work with. Let’s store that value for future use by redirecting it into a file:

At this point, you have an original file. Copy that file to the /tmp directory with the name duplicate.txt . Copy the file by using the following command (be sure to copy, not move):

Run the following command to create a checksum of the copied file:

Next, append the hash result to our hashes.txt file and then compare the two. Be very careful to use the >> append redirect operator here, because > will overwrite the hash value of the original.txt file.

Run the following command:

The two hash results are identical, so the file did not change during the copy process.

Next, simulate a change. Type the following command to change the /tmp/duplicate.txt file contents, and then rerun the md5sum command with the >> append operator:

You know that the duplicate.txt file is no longer identical to the original.txt file, but let’s prove that:

The two checksum values are not identical, and therefore the two files from which the checksums were generated are not identical.

In the above example, you manually compared the hash values by displaying them with cat . You can use the —check option to have md5sum do the comparison for us. I’ve included both methods below:

You can repeat the above steps substituting sha256sum for the md5sum command to see how the process works using the SHA algorithm. The sha256sum command also includes a —check checksum option that compares the resulting hashes and displays a message for whether the files differ.

Note: If you transfer files between Linux, macOS, and Windows, you can still use hashing to verify the files’ integrity. To generate a hash value on macOS, run the md5 command. To do this in Windows, you must download a third party program. Personally, I use md5checker. Be sure to understand licensing for these utilities. You may be able to use the PowerShell cmdlet get-filehash , depending on the version of PowerShell you have installed.

[ Free course: Red Hat Satellite Technical Overview. ]

Wrap up

Hashing confirms that data has not unexpectedly changed during a file transfer, download, or other event. This concept is known as file integrity. Hashing does not tell you what changed, just that something changed. Once hashing tells you two files are different, you can use commands such as diff to discover what differences exist.

Источник