I am trying to create a pretty basic Glue job. I have two different AWS RDS Mariadb’s, with two similar tables (field names are different). I would like to transform the data from table A so it fits with table B schema (this seems pretty trivial and is working). And then i would like to update all exist…
Tag: sql
difference between autobegin and Session.begin
In the following code snippet a commit is executed once the with block is exited. As per the docs https://docs.sqlalchemy.org/en/20/orm/session_api.html#sqlalchemy.orm.Session.begin The Session object features autobegin behavior, so that normally it is not necessary to call the Session.begin() method explicit…
Inserting csv file into a database using Python
In Python I’ve connected to a Postgres database using the following code: I have created a table called departments and want to insert data into the database from a CSV file. I read the csv in as follows: And I am trying to insert this data into the table with the following code: which I’ve seen d…
How to execute sql query with python variables null or empty
Hi I’m using python 3 and snowflake. Example: I tried with this: I’m getting TypeError: not enough arguments for format string. I would like to run this query for all schemas if I dont have defined MY_SCHEMA variable and also for all tables if MY_TABLE variable is null or empty. I would be gratefu…
convert data to 1NF
I have a dataset like this name date singer language phase 1 Yes or No 02.01.20 Benjamin Smith en 1 2 Parabens 01.06.21 Rafael Galvao;Simon Murphy pt;en 2 3 Love 12.11.20 Michaela Condell en 1 4 Paz 11.07.19 Ana Perez; Eduarda Pinto es;pt 3 5 Stop 12.01.21 Michael Conway;Gabriel Lee en;en 1 6 Shalom 18.06.21 …
Pass/Refer a SQL file in Apache Beam instead of string
I’m trying to run a simple Beam pipeline to extract data from a BQ table using SQL and push to a GCS bucket. My requirement is to pass the SQL from a file (a simple .sql file) and not as a string. I want to modularize the SQL. So far, I’ve tried the following option – it did not work:
Regexp_replace “,” with “.” every other commas in spark
I have a dataframe that instead of . it has , and separators of numbers are also comma, I need to replace only odd comma to dot. The dataframe is very big but as an example, I have this: I want this df: Answer You can split on all commas , and later you can use for-loop: with range(0, len(splitted_data),
how to use multi variables in a single SQL python code
I have used this code to use two variables in a single SQL code in Python : but I’ve got this error : MySQLInterfaceError: Python type tuple cannot be converted though I’ve converted my strings into a tuple like this: how can I use these two variables in my cursor.execute code? Answer
How to union two counts queries in SQLAlchemy?
I have two queries and the only difference between then is that one is counting the success status and the other failure status. Is there a way to get this result in just one query? I’m using SQLALchemy to do the queries. Answer You can use conditions on Count, your query will look like:
I have problems with Python sqlite3 library, for protections against SQL Injections
I’m trying to prevent SQL injections with code in Python, using library sqlite3. I went accross all websites in existence concerning this issue, but none of them are fixing my problem. This statement is here is executed via an email_in_use(email:str) function, to check if an email does exist in the cred…