Skip to content
Advertisement

Lost files while tried to move files using python shutil.move

I had 120 files in my source folder which I need to move to a new directory (destination). The destination is made in the function I wrote, based on the string in the filename. For example, here is the function I used.

path   ='/path/to/source'
dropbox='/path/to/dropbox'
files = = [os.path.join(path,i).split('/')[-1] for i in os.listdir(path) if i.startswith("SSE")]
sam_lis      =list()
for sam in files:
    sam_list =sam.split('_')[5]
    sam_lis.append(sam_list)
    sam_lis  =pd.unique(sam_lis).tolist()

# Using the above list
ID  = sam_lis

def filemover(ID,files,dropbox):
    """
    Function to move files from the common place to the destination folder
    """
    for samples in ID:
        for fs in files:
            if samples in fs:
                desination = dropbox  + "/"+ samples + "/raw/"
                if not os.path.isdir(desination):
                    os.makedirs(desination)
        for rawfiles in fnmatch.filter(files, pat="*"):
            if samples in rawfiles:
                shutil.move(os.path.join(path,rawfiles), 
                            os.path.join(desination,rawfiles))

In the function, I am creating the destination folders, based on the ID’s derived from the files list. When I tried to run this for the first time it threw me FILE NOT exists error. However, later when I checked the source all files starting with SSE were missing. In the beginning, the files were there. I want some insights here;

  1. Whether or not os.shutil.move moves the files to somewhere like a temp folder instead of destination folder?
  2. whether or not the os.shutil.move deletes the files from the source in any circumstance?
  3. Is there any way I can test my script to find the potential reasons for missing files?

Any help or suggestions are much appreciated?

Advertisement

Answer

It is late but people don’t understand the op’s question. If you move a file into a non-existing folder, the file seems to become a compressed binary and get lost forever. It has happened to me twice, once in git bash and the other time using shutil.move in Python. I remember the python happens when your shutil.move destination points to a folder instead of to a copy of the full file path.

For example, if you run the code below, a similar situation to what the op described will happen:

src_folder = r'C:/Users/name'
dst_folder = r'C:/Users/name/data_images'
file_names = glob.glob(r'C:/Users/name/*.jpg')

for file in file_names:
    file_name = os.path.basename(file)
    shutil.move(os.path.join(src_folder, file_name), dst_folder)

Note that dst_folder in the else block is just a folder. It should be dst_folder + file_name. This will cause what the Op described in his question. I find something similar on the link here with a more detailed explanation of what went wrong: File moving mistake with Python

Advertisement