I am trying to create a pretty basic Glue job. I have two different AWS RDS Mariadb’s, with two similar tables (field names are different). I would like to transform the data from table A so it fits with table B schema (this seems pretty trivial and is working). And then i would like to update all existing entries (on
Tag: sql
difference between autobegin and Session.begin
In the following code snippet a commit is executed once the with block is exited. As per the docs https://docs.sqlalchemy.org/en/20/orm/session_api.html#sqlalchemy.orm.Session.begin The Session object features autobegin behavior, so that normally it is not necessary to call the Session.begin() method explicitly Running without begin, like so does not commit even though the docs mention autobegin is not needed. The docs for Session.begin
Inserting csv file into a database using Python
In Python I’ve connected to a Postgres database using the following code: I have created a table called departments and want to insert data into the database from a CSV file. I read the csv in as follows: And I am trying to insert this data into the table with the following code: which I’ve seen done in various articles
How to execute sql query with python variables null or empty
Hi I’m using python 3 and snowflake. Example: I tried with this: I’m getting TypeError: not enough arguments for format string. I would like to run this query for all schemas if I dont have defined MY_SCHEMA variable and also for all tables if MY_TABLE variable is null or empty. I would be grateful for any advice. Answer I found
convert data to 1NF
I have a dataset like this name date singer language phase 1 Yes or No 02.01.20 Benjamin Smith en 1 2 Parabens 01.06.21 Rafael Galvao;Simon Murphy pt;en 2 3 Love 12.11.20 Michaela Condell en 1 4 Paz 11.07.19 Ana Perez; Eduarda Pinto es;pt 3 5 Stop 12.01.21 Michael Conway;Gabriel Lee en;en 1 6 Shalom 18.06.21 Shimon Cohen hebr 1 7
Pass/Refer a SQL file in Apache Beam instead of string
I’m trying to run a simple Beam pipeline to extract data from a BQ table using SQL and push to a GCS bucket. My requirement is to pass the SQL from a file (a simple .sql file) and not as a string. I want to modularize the SQL. So far, I’ve tried the following option – it did not work:
Regexp_replace “,” with “.” every other commas in spark
I have a dataframe that instead of . it has , and separators of numbers are also comma, I need to replace only odd comma to dot. The dataframe is very big but as an example, I have this: I want this df: Answer You can split on all commas , and later you can use for-loop: with range(0, len(splitted_data),
how to use multi variables in a single SQL python code
I have used this code to use two variables in a single SQL code in Python : but I’ve got this error : MySQLInterfaceError: Python type tuple cannot be converted though I’ve converted my strings into a tuple like this: how can I use these two variables in my cursor.execute code? Answer
How to union two counts queries in SQLAlchemy?
I have two queries and the only difference between then is that one is counting the success status and the other failure status. Is there a way to get this result in just one query? I’m using SQLALchemy to do the queries. Answer You can use conditions on Count, your query will look like:
I have problems with Python sqlite3 library, for protections against SQL Injections
I’m trying to prevent SQL injections with code in Python, using library sqlite3. I went accross all websites in existence concerning this issue, but none of them are fixing my problem. This statement is here is executed via an email_in_use(email:str) function, to check if an email does exist in the creditentials table (I know I could use count to do