Skip to content

Read avro files in pyspark with PyCharm

I’m quite new to spark, I’ve imported pyspark library to pycharm venv and write below code: , everything seems to be okay but when I want to read avro file I get message: pyspark.sql.utils.AnalysisException: ‘Failed to find data source: avro. Avro is built-in but external data source module …

Poetry and PyTorch

I’ve recently found poetry to manage dependencies. In one project, we use PyTorch. How do I add this to poetry? We are working on machines that have no access to a CUDA GPU (for simple on the road inferencing/testing) and workstations where we do have access to CUDA GPUs. Is it possible to use poetry to…