Skip to content
Advertisement

Reading a file without locking it in Python

I want to read a file, but without any lock on it.

with open(source, "rb") as infile:
            data = infile.read()

Can the code above lock the source file?

This source file can be updated at any time with new rows (during my script running for example). I think not because it is only in reading mode (“rb”). But I found that we can use Windows API to read it without lock. I did not find an simple answer for my question.

My script runs locally but the source file and the script/software which appends changes on it are not (network drive).

Advertisement

Answer

Opening a file does not put a lock on it. In fact, if you needed to ensure that separate processes did not access a file simultaneously, all these processes would have to to cooperatively take special steps to ensure that only a single process accessed the file at one time (see Locking a file in Python). This can also be demonstrated by the following small program that purposely takes its time in reading a file to give another process (namely me with a text editor) a chance to append some data to the end of the file while the program is running. This program reads and outputs the file one byte at a time pausing .1 seconds between each read. During the running of the program I added some additional text to the end of the file and the program printed the additional text:

import time


with open('test.txt', "rb") as infile:
    while True:
        data = infile.read(1)
        if data == b'':
            break
        time.sleep(.1)
        print(data.decode('ascii'), end='', flush=True)

You can read your file in pieces and then join these pieces together if you need one single byte string. But this will not be as memory efficient as reading the file with a single read:

BLOCKSIZE = 64*1024 # or some other value depending on the file size
with open(source, "rb") as infile:
    blocks = []
    while True:
        data = infile.read(BLOCKSIZE)
        if data == b'':
            break
        blocks.append(data)
# if you need the data in one piece (otherwise the pieces are in blocks):
data = b''.join(blocks)
User contributions licensed under: CC BY-SA
8 People found this is helpful
Advertisement