The Python tempfile Module
Temporary files, or "tempfiles", are mainly used to store intermediate information on disk for an application. These files are normally created for different purposes such as temporary backup or if the application is dealing with a large dataset bigger than the system's memory, etc. Ideally, these files are located in a separate directory, which varies on different operating systems, and the name of these files are unique. The data stored in temporary files is not always required after the application quits, so you may want these files to be deleted after use.
Python provides a module known as tempfile, which makes creating and handling temporary files easier. This module provides a few methods to create temporary files and directories in different ways.
tempfile comes in handy whenever you want to use temporary files to store data in a Python program. Let's take a look at a couple of different examples on how the
tempfile module can be used.
Creating a Temporary File
Suppose your application needs a temporary file for use within the program, i.e. it will create one file, use it to store some data, and then delete it after use. To achieve this, we can use the
This function will create one temporary file to the default
tempfile location. This location may be different between operating systems. The best part is that the temporary file created by
TemporaryFile() will be removed automatically whenever it the file is closed. Also, it does not create any reference to this file in the system's filesystem table. This makes it private to the current application i.e. no other program will be able to open the file.
Let's take a look at the below Python program to see how it works:
import tempfile #1 print("Creating one temporary file...") temp = tempfile.TemporaryFile() #2 try: print("Created file is:", temp) #3 print("Name of the file is:", temp.name) #4 finally: print("Closing the temp file") temp.close() #5
It will print the below output:
$ python3 temp-file.py Creating one temporary file... Created file is: <_io.BufferedRandom name=4> Name of the file is: 4 Closing the temp file
- To create one temporary file in Python, you need to import the
- As explained above, we have created the temporary file using the
- From the output, you can see that the created object is actually not a file, it is a file-like object. And, the
modeparameter (not shown in our example) of the created file is
w+b, i.e. you can read and write without being closed.
- The temporary file created has no name.
- Finally, we are closing the file using the
close()method. It will be destroyed after it is closed.
One thing we should point out is that the file created using the
TemporaryFile() function may or may not have a visible name in the file system. On Unix, the directory entry for the file is removed automatically after it is created, although this is not supported on other platforms. Normally
TemporaryFile() is the ideal way to create one temporary storage area for any program in Python.
Create a Named Temporary File
In our previous example, we have seen that the temporary file created using the
TemporaryFile() function is actually a file-like object without an actual file name. Python also provides a different method,
NamedTemporaryFile(), to create a file with a visible name in the file system. Other than providing a name to the tempfile,
NamedTemporaryFile() works the same as
TemporaryFile(). Now let's use the same above example to create a named temporary file:
import tempfile print("Creating one named temporary file...") temp = tempfile.NamedTemporaryFile() try: print("Created file is:", temp) print("Name of the file is:", temp.name) finally: print("Closing the temp file") temp.close()
Running this code will print output similar to the following:
$ python3 named-temp-file.py Creating one named temporary file... Created file is: <tempfile._TemporaryFileWrapper object at 0x103f22ba8> Name of the file is: /var/folders/l7/80bx27yx3hx_0_p1_qtjyyd40000gn/T/tmpa3rq8lon Closing the temp file
So, the created file actually has a name this time. The advantage of
NamedTemporaryFile() is that we can save the name of the created temp files and use them later before closing or destroying it. If the
delete parameter is set to
False, then we can close the file without it being destroyed, allowing us to re-open it later on.
Providing a Suffix or Prefix to the Name
Sometimes we need to add a prefix or suffix to a temp-file's name. It will help us to identify all temp files created by our program.
To achieve this, we can use the same
NamedTemporaryFile function defined above. The only thing we need to add is two extra parameters while calling this function:
import tempfile temp = tempfile.NamedTemporaryFile(prefix="dummyPrefix_", suffix="_dummySuffix") try: print("Created file is:", temp) print("Name of the file is:", temp.name) finally: temp.close()
Running this code will print the following output:
$ python3 prefix-suffix-temp-file.py Created file is: <tempfile._TemporaryFileWrapper object at 0x102183470> Name of the file is: /var/folders/tp/pn3dvz_n7cj7nfs0y2szsk9h0000gn/T/dummyPrefix_uz63brcp_dummySuffix
So, if we will pass the two extra arguments
prefix to the
NamedTemporaryFile() function, it will automatically add those in the start and end of the file name.
Finding the Default Location of Temp Files
tempfile.tempdir variable holds the default location for all temporary files. If the value of
None or unset, Python will search a standard list of directories and sets
tempdir to the first directory value, but only if the calling program can create a file in it. The following are the list of directories it will scan, in this order:
- The directory named by the TMPDIR environment variable.
- The directory named by the TEMP environment variable.
- The directory named by the TMP environment variable
- Platform-specific directories:
- On windows, C:\TEMP, C:\TMP, \TEMP, and \TMP, in the same order.
- On other platforms, /tmp, /var/tmp, and /usr/tmp, in the same order.
- The current working directory.
To find out the default location of temporary files, we can call
tempfile.gettempdir() method. It will return the value of
tempdir if it is not
None. Otherwise it will first search for the directory location using the steps mentioned above and then return the location.
import tempfile print("Current temp directory:", tempfile.gettempdir()) tempfile.tempdir = "/temp" print("Temp directory after change:", tempfile.gettempdir())
If you will run the above program, it will print an output similar to the following:
$ python3 dir-loc-temp-file.py Current temp directory: /var/folders/tp/pn3dvz_n7cj7nfs0y2szsk9h0000gn/T Temp directory after change: /temp
You can see that the first temp directory location is the system-provided directory location and the second temp directory is the same value as the one that we have defined.
Reading and Writing Data from Temp Files
We have learned how to create a temporary file, create a temporary file with a name, and how to create a temporary file with a suffix and/or prefix. Now, let's try to understand how to actually read and write data from a temporary file in Python.
Reading and writing data from a temporary file in Python is pretty straightforward. For writing, you can use the
write() method and for reading, you can use the
read() method. For example:
import tempfile temp = tempfile.TemporaryFile() try: temp.write(b'Hello world!') temp.seek(0) print(temp.read()) finally: temp.close()
This will print the output as
b'Hello world!' since the
write() method takes input data in bytes (hence the
b prefix on the string).
If you want to write text data into a temp file, you can use the
writelines() method instead. For using this method, we need to create the tempfile using
w+t mode instead of the default
w+b mode. To do this, a
mode param can be passed to
TemporaryFile() to change the mode of the created temp file.
import tempfile temp = tempfile.TemporaryFile(mode='w+t') try: temp.writelines("Hello world!") temp.seek(0) print(temp.read()) finally: temp.close()
Unlike the previous example, this will print "Hello World" as the output.
Create a Temporary Directory
If your program has several temporary files, it may be more convenient to create one temporary directory and put all of your temp files inside of it. To create a temporary directory, we can use the
TemporaryDirectory() function. After all temp files are closed, we need to delete the directory manually.
import tempfile with tempfile.TemporaryDirectory() as tmpdirname: print('Created temporary directory:', tmpdirname) # Both the directory and its contents have been deleted
It will print the below output:
$ python3 mk-dir-temp-file.py Created temporary directory: /var/folders/l7/80bx27yx3hx_0_p1_qtjyyd40000gn/T/tmpn_ke7_rk
Create a Secure Temporary File and Directory
mkstemp(), we can create a temporary file in the most secure manner possible. The temporary file created using this method is readable and writable only by the creating user ID. We can pass
suffix arguments to add prefix and suffix to the created file name. By default, it opens the file in binary mode. To open it in text mode, we can pass
text=True as an argument to the function. Unlike
TemporaryFile(), the file created by
mkstemp() doesn't get deleted automatically after closing it.
As you can see in the example below, the user is responsible for deleting the file.
import tempfile import os temp_directory = tempfile.TemporaryDirectory() print("Directory name:", temp_directory) os.removedirs(temp_directory)
$ python3 mk-secure-dir-temp-file.py Directory name: /var/folders/tp/pn3dvz_n7cj7nfs0y2szsk9h0000gn/T/tmpf8f6xc53
mkstemp(), we can create a temporary directory in the most secure manner possible using
mkdtemp() method. And again, like
mkstemp(), it also supports
suffix arguments for adding a prefix and suffix to the directory name.
In this article we have learned different ways to create temporary files and directories in Python. You can use temp files in any Python program you want. But just make sure to delete it if the particular method used doesn't automatically delete it on its own. Also keep in mind that behavior may different between operating systems, like the output directory names and file names.
All of these functions we have explained above works with many different arguments, although we have not covered in detail what type of arguments each function takes. If you want to learn more about the
tempfile module, you should check out the Python 3 official documentation.