I have been trying to create an excel file with several sheets from delta tables, however some of my column names include _x10e6 _ which is apparently translated to ღ. I have tried to use encoding=’unicode_escape’ and encoding=’utf-8′ without luck. I cannot use xlsxwriter because I am appending to an existing file. Does anybody know how I can keep _x10e6
Tag: delta-lake
Double assignment of same variable in one expression in Python – does it have any purpose?
I was going through delta lake documentation page. There is a line like this : In the last line, we see a double assignment of same variable (spark). Does this do something different compared to : In general, in python, is there a meaning to double assignment of same variable ? Answer Short answer: It’s meaningless. Aside from a few
Correct Method to Delete Delta Lake Partion on AWS s3
I need to delete a Delta Lake partition with associated AWS s3 files and then need to make sure AWS Athena displays this change. The purpose is because I need to rerun some code to re-populate the data. I tried this And it completed with no errors but the files on s3 still exist and Athena still shows the data
How to write data to Delta Lake from Kubernetes
Our organisation runs Databricks on Azure that is used by data scientists & analysts primarily for Notebooks in order to do ad-hoc analysis and exploration. We also run Kubernetes clusters for non spark-requiring ETL workflows. We would like to use Delta Lakes as our storage layer where both Databricks and Kubernetes are able to read and write as first class