When I run the following code on the server (I think, when reading the json file with google application credentials), I get the following error: Json as( source here ): My Code: When I run it with spyder IDE on my local computer with windows installed, there is no problem. Answer I solved the problem. The python version installed on

getting, TypeError: json cannot serialize object of type ‘byte. When loading table to big query via server

When I run the following code on the server (I think, when reading the json file with google application credentials), I get the following error:

TypeError: Object of type 'bytes' is not JSON serializable

Json as( source here ):

My Code:

import mysql.connector
from google.cloud import bigquery
import pandas as pd
import os
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="/root/google_credentials.json"
def incomes():

  client = bigquery.Client()
  print("connecting MySql...")
  mydb = mysql.connector.connect(
            host='host',
          user='1111',
          password='password',
          database="my_database")
          
  print ("Connection ok")     
  mycursor = mydb.cursor()
  mycursor.execute('''query''')
        
  myresult = mycursor.fetchall()
  df=pd.DataFrame(myresult,columns=mycursor.column_names)


  dataset_ref = client.dataset('dataset')    
  job_config = bigquery.LoadJobConfig()
  job_config.autodetect = True
  job_config.write_disposition = "WRITE_TRUNCATE"
  load_job = client.load_table_from_dataframe(
            df, 'table_name', job_config=job_config
  )
  print("Starting job {}".format(load_job))
  return ("Done!", 200)
       
incomes()

When I run it with spyder IDE on my local computer with windows installed, there is no problem.

Answer

I solved the problem. The python version installed on the server was 3.6.8. I upgraded this version to python 3.8.12 and the problem was solved.

Advertisement

Answer