Python: how to create a new file and write contents of variable to it?

I am writing a program that outputs the file types inside a directory by looking at their headers.

Some of the files are compressed so I need to be able to decompress them as a starting point

So far I have been able to search directories and using the header change the extensions, and open the compressed file and store its contents in a variable, now I am having trouble saving the variable as a new file.

I’m guessing I will have to use the a command along the lines of f.write(«newfile», content) but not sure.

Thanks in advance

3 Answers 3

In general, if you have a string in a variable foo , you can write it to a file with:

In your case, you wouldn’t use f as you’re already using f for your input file handle.

I suppose you’d want something like:

you should do something like:

You do not have to look at the first two bytes to identify gz files. Instead, I think a more «Pythonic» approach would be to try first, apologize later (more commonly known as «Easier to ask Forgiveness than Permission»):

Also, it is better to avoid using os.chdir if possible because it alters the current directory even after you leave the uncompress function. If your script deals with other directories, then you have to carefully control what the current directory is at every stage of your program. If you use os.path.join instead, you never have to worry about what is the current directory.

Related

Hot Network Questions

Subscribe to RSS

To subscribe to this RSS feed, copy and paste this URL into your RSS reader.

site design / logo © 2021 Stack Exchange Inc; user contributions licensed under cc by-sa. rev 2021.4.16.39093

By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.

Reading and Writing Files in Python

Last Updated: February 5, 2021

Overview

When you’re working with Python, you don’t need to import a library in order to read and write to files. It’s handled natively in the language, albeit in a unique manner.

The first thing you’ll need to do is use the built-in python open file function to get a file object.

The open function opens a file. It’s simple. This is the first step in reading and writing files in python.

When you use the open function, it returns something called a file object. File objects contain methods and attributes that can be used to collect information about the file you opened. They can also be used to manipulate said file.

For example, the mode attribute of a file object tells you which mode a file was opened in. And the name attribute tells you the name of the file that the file object has opened.

You must understand that a file and file object are two wholly separate – yet related – things.

File Types

What you may know as a file is slightly different in Python.

In Windows, for example, a file can be any item manipulated, edited or created by the user/OS. That means files can be images, text documents, executables, and excel file and much more. Most files are organized by keeping them in individual folders.

In Python, a file is categorized as either text or binary, and the difference between the two file types is important.

Text files are structured as a sequence of lines, where each line includes a sequence of characters. This is what you know as code or syntax.

Each line is terminated with a special character, called the EOL or End of Line character. There are several types, but the most common is the comma <,>or newline character. It ends the current line and tells the interpreter a new one has begun.

A backslash character can also be used, and it tells the interpreter that the next character – following the slash – should be treated as a new line. This character is useful when you don’t want to start a new line in the text itself but in the code.

A binary file is any type of file that is not a text file. Because of their nature, binary files can only be processed by an application that know or understand the file’s structure. In other words, they must be applications that can read and interpret binary.

Open ( ) Function

In order to begin reading and writing files in python, you must rely on the built-in python open file function. Use open() to open a file in python.

As explained above, open ( ) will return a file object, so it is most commonly used with two arguments.

Recommended Python Training

For Python training, our top recommendation is DataCamp.

An argument is nothing more than a value that has been provided to a function, which is relayed when you call it. So, for instance, if we declare the name of a file as “Test File,” that name would be considered an argument.

The second argument you see – mode – tells the interpreter and developer which way the file will be used.

Including a mode argument is optional because a default value of ‘r’ will be assumed if it is omitted. The ‘r’ value stands for read mode, which is just one of many.

• ‘r’ – Python read file. Read mode which is used when the file is only being read
• ‘w’ – Python write file. Write mode which is used to edit and write new information to the file (any existing files with the same name will be erased when this mode is activated)
• ‘a’ – Python append file. Append mode, which is used to add new data to the end of the file; that is new information is automatically amended to the end
• ‘r+’ – Special read and write mode, which is used to handle both actions when working with a file

So, let’s take a look at a quick example where we write file named “workfile” using the python open method.

This snippet opens the file named “workfile” in writing mode so that we can make changes to it. Be sure you have the correct file path. The current information stored within the file is also displayed – or printed – for us to view.

Once this has been done, you can move on to call the objects functions. The two most common functions are read and write.

Python Write To File

To get more familiar with text files in Python, let’s create our own and do some additional exercises.

Using a simple text editor, let’s create a file. You can name it anything you like, and it’s better to use something you’ll identify with. In this example, we will use a txt file type , but you could also create a csv file .

For the purpose of this tutorial, however, we are going to call the new file “testfile.txt”.

Once the new file is created, it will be blank. The next step is to add content using the python write method.

To manipulate the file, write the following in your Python environment (you can copy and paste if you’d like):

Naturally, if you open the text file – or look at it – using Python you will see only the text we told the interpreter to add. We didn’t add any newline characters above, but you certainly can “\n” to format your file when viewing.

Reading a File in Python

There are actually a number of ways to read a text file in Python, not just one.

If you need to extract a string that contains all characters in the file, you can use the following python file operation:

The full code to work with this method will look something like this:

Please be sure you have the correct file path, whether it is relative path or absolute path, or your python program will fail to open the file . The output of that command will display all the text inside the file, the same text we told the interpreter to add earlier. There’s no need to write it all out again, but if you must know, everything will be shown except for the “$ cat testfile.txt” line.

Another way to read a file is to call a certain number of characters.

For example, with the following code the interpreter will read the first five characters of stored data and return it as a string:

Notice how we’re using the same file.read() method, only this time we specify the number of characters to process from the input file.

The output for this will look like:

If you want to read a file line by line – as opposed to pulling the content of the entire file at once – then you use the readline() function.

Why would you use something like this?

Let’s say you only want to see the first line of the file – or the third. You would execute the readline() function as many times as possible to get the data you were looking for.

Each time you run the method, it will return a string of characters that contains a single line of information from the file.

This would return the first line of the file, like so:

If we wanted to return only the third line in the file, we would use this:

But what if we wanted to return every line in the file, properly separated? You would use the same function, only in a new form. This is called the file.readlines() function.

The output you would get from this is:

Notice how each line is separated accordingly? Note that this is not the ideal way to show users the content in a file. But it’s great when you want to collect information quickly for personal use during development or recall.

Looping over a file object

When you want to read – or return – all the lines from a file in a more memory efficient, and fast manner, you can use the loop over method. The advantage to using this method is that the related code is both simple and easy to read.

This will return:

See how much simpler that is than the previous methods?

Using the File Write Method

One thing you’ll notice about the file write method is that it only requires a single parameter, which is the string you want to be written.

This method is used to add information or content to an existing file. To start a new line after you write data to the file, you can add an EOL character.

Obviously, this will amend our current file to include the two new lines of text. There’s no need to show output.

Closing a File

When you’re done working, you can use the fh.close() command to end things. What this does is close the file completely, terminating resources in use, in turn freeing them up for the system to deploy elsewhere.

It’s important to understand that when you use the fh.close() method, any further attempts to use the file object will fail.

Notice how we have used this in several of our examples to end interaction with a file? This is good practice.

File Handling in Python

To help you better understand some of the file handling methods discussed here, we’re going to offer a python script with real world examples of each use. Feel free to copy the code and try it out for yourself in a Python interpreter (make sure you have any named files created and accessible first).

Opening a text file:

Reading a text file:

To read a text file one line at a time:

To read a list of lines in a text file:

To write new content or text to a file:

You can also use this to write multiple lines to a file at once(python newline example):

To append a file:

To close a file completely when you are done:

With Statement

You can also work with file objects using the with statement. It is designed to provide much cleaner syntax and exceptions handling when you are working with code. That explains why it’s good practice to use the with statement where applicable.

One bonus of using this method is that any files opened will be closed automatically after you are done. This leaves less to worry about during cleanup.

To use the with statement to open a file:

Now that you understand how to call this statement, let’s take a look at a few examples.

You can also call upon other methods while using this statement. For instance, you can do something like loop over a file object:

You’ll also notice that in the above example we didn’t use the “file.close()” method because the with statement will automatically call that for us upon execution. It really makes things a lot easier, doesn’t it?

Using the With Statement in Python

To better understand the with statement, let’s take a look at some real world examples just like we did with the file functions.

To write to a file using the with statement:

To read a file line by line, output into a list:

This will take all of the text or content from the “hello.txt” file and store it into a string called “data”.

Splitting Lines in a Text File

As a final example, let’s explore a unique function that allows you to split the lines taken from a text file. What this is designed to do, is split the string contained in variable data whenever the interpreter encounters a space character.

But just because we are going to use it to split lines after a space character, doesn’t mean that’s the only way. You can actually split your text using any character you wish – such as a colon, for instance.

The code to do this (also using a with statement) is:

If you wanted to use a colon instead of a space to split your text, you would simply change line.split() to line.split(“:”).

The output for this will be:

The reason the words are presented in this manner is because they are stored – and returned – as an array. Be sure to remember this when working with the split function.

Python Write File/ Read File

This tutorial covers the following topic – Python Write File/Read File. It describes the syntax of the writing to a file in Python. Also, it explains how to write to a text file and provides several examples for help.

For writing to a file in Python, you would need a couple of functions such as Open(), Write(), and Read(). All these are built-in Python functions and don’t require a module to import.

There are majorly two types of files you may have to interact with while programming. One is the text file that contains streams of ASCII or UNICODE (UTF-8) characters. Each line ends with a newline (“\n”) char, a.k.a. EOL (End of Line).

Another type of file is called binary that contains machine-readable data. It doesn’t have, so-called line as there is no line-ending. Only the application using it would know about its content.

Anyways, this tutorial will strictly tell you to work with the text files only.

Python Write File Explained with Examples

Let’s begin this tutorial by taking on the first call required to write to a file in Python, i.e., Open().

Open File in Python

You first have to open a file in Python for writing. Python provides the built-in open() function.

The open() function would return a handle to the file if it opened successfully. It takes two arguments, as shown below:

The first argument is the name or path of the file (including file name). For example – sample_log.txt or /Users/john/home/sample_log.txt.

And the second parameter (optional) represents a mode to open the file. The value of the “access_mode” defines the operation you want to perform on it. The default value is the READ only mode.

File Open Modes

It is optional to pass the mode argument. If you don’t set it, then Python uses “r” as the default value for the access mode. It means that Python will open a file for read-only purpose.

However, there are a total of six access modes available in python.

• “r” – It opens a text file for reading. It keeps the offset at the start of the file. If the file is missing, then it raises an I/O error. It is also the default mode.
• “r+” – It opens the file for both READ and WRITE operations. It sets the offset at the start of the file. An I/O error occurs for a non-existent file.
• “w” – It opens a file for writing and overwrites any existing content. The handle remains at the start of the data. If the file doesn’t exist, then it creates one.
• “w+” – It opens the file for both READ and WRITE operations. Rest, it works the same as the “w” mode.
• “a” – It opens a file for writing or creates a new one if the file not found. The handle moves to the end (EOF). It preserves the existing content and inserts data to the end.
• “a+” – It opens the file for both READ and WRITE operations. Rest, it works the same as the “a” mode.

Check out a few examples:

Write File in Python

Python provides two functions to write into a text file: Write() and Writelines().

1. write() – Let’s first use write() for writing to a file in Python. This function puts the given text in a single line.

But, first, open any IDE and create a file named “sample_log.txt” for our test. Don’t make any other changes to it.

Please note – If you try to open a file for reading and it doesn’t exist, then Python will throw the FileNotFoundError exception.

To edit this file from your Python program, we’ve given the following code:

We’ve opened the file in “w” mode, which means to overwrite anything written previously. So, after you open it and see its content, you’ll find the new text in four different lines.

2. writelines() – The writelines() function takes a list of strings as the input and inserts each of them as a separate line in one go. You can check its syntax below:

Append File in Python

You also need to know how to append the new text to an existing file. There are two modes available for this purpose: a and a+.

Whenever you open a file using one of these modes, the file offset is set to the EOF. So, you can write the new content or text next to the existing content.

Let’s understand it with a few lines of code:

We’ll first open a file in “a” mode. If you run this example the first time, then it creates the file.

So far, two lines have been added to the file. The second write operation indicates a successful append.

Now, you’ll see the difference between the “a” and “a+” modes. Let’s try a read operation and see what happens.

The above line of code would fail as the “a” mode doesn’t allow READ. So, close it, open, and then do a read operation.

The output is something like:

Let’s now try appending using the “a+” mode. Check out the below code:

Read File in Python

For reading a text file, Python bundles the following three functions: read(), readline(), and readlines()

1. read() – It reads the given no. of bytes (N) as a string. If no value is given, then it reads the file till the EOF.

2. readline() – It reads the specified no. of bytes (N) as a string from a single line in the file. It restricts to one line per call even if N is more than the bytes available in one line.

3. readlines() – It reads every line presents in the text file and returns them as a list of strings.

It is so easy to use the Python read file functions that you can check yourself. Just type the following code in your IDE or the default Python IDE, i.e., IDLE.

Please note that the Python seek() function is needed to change the position of file offset. It decides the point to read or write in the file. Whenever you do a read/write operation, it moves along.

Close File in Python

File handling in Python starts with opening a file and ends with closing it. It means that you must close a file after you are finished doing the file operations.

Closing a file is a cleanup activity, which means to free up the system resources. It is also essential because you can only open a limited number of file handles.

Also, please note that any attempt to access the file after closing would throw an I/O error. You may have already seen us using it in our previous examples in this post.

With Statement in Python

If you want a cleaner and elegant way to write to a file in Python, then try using the WITH statement. It does the automatic clean up of the system resources like file handles.

Also, it provides out of the box exception handling (Python try-except) while you are working with the files. Check out the following example to see how with statement works.

Working sample code

Below is a full-fledged example that demonstrates the usage of the following functions:

• Python Write file using write() & writelines()
• Python Read file operations using read(), readline(), and readlines()

This Python program generates the following output:

Attempt the Quiz

We’ve now come to the closure of this Read/Write File in Python tutorial. If you have read it from the start to end, then file handling would be on your tips. However, we recommend the following quizzes to attempt.

These are quick questionnaires to test how much knowledge have you retained after reading the above stuff.

Also, you must use these concepts in your projects or can also write some hands-on code to solve real-time problems. It’ll certainly help you grasp faster and remember better.

By the way, if you wish to learn Python from scratch to depth, then do read our step by step Python tutorial .