Write a CSV file asynchronously in Python

Tags: , , , ,



I am writing a CSV file with the following function:

import csv
import os
import aiofiles


async def write_extract_file(output_filename: str, csv_list: list):
    """
    Write the extracted content into the file
    """
    try:
        async with aiofiles.open(output_filename, "w+") as csv_file:
            writer = csv.DictWriter(csv_file, fieldnames=columns.keys())
            writer.writeheader()
            writer.writerows(csv_list)
    except FileNotFoundError:
        print("Output file not present", output_filename)
        print("Current dir: ", os.getcwd())
        raise FileNotFoundError

However, as there is no await allowed over writerows method, there are no rows being written into the CSV file.
How to resolve this issue? Is there any workaround available?
Thank you.
Entire code can be found here.

Answer

In my opinion it’s better not to try to use the aiofiles with the csv module and run the synchronous code using loop.run_in_executor and wait it asynchronously like below:

def write_extract_file(output_filename: str, csv_list: list):
    """
    Write the extracted content into the file
    """
    try:
        with open(output_filename, "w+") as csv_file:
            writer = csv.DictWriter(csv_file, fieldnames=columns.keys())
            writer.writeheader()
            writer.writerows(csv_list)
    except FileNotFoundError:
        print("Output file not present", output_filename)
        print("Current dir: ", os.getcwd())
        raise FileNotFoundError


async def main():
    loop = asyncio.get_running_loop()
    await loop.run_in_executor(None, write_extract_file, 'test.csv', csv_list)


Source: stackoverflow