I’m trying to take a query of recent customer transactions and match potential primary phone, cellphone and work phone matches against a particular list of customers I have. Essentially, I am taking one dataframe column (the list of customers I am trying to see if they had transactions recently) against…
Tag: python
IndexError: tensors used as indices must be long, byte or bool tensors
I am getting this error only during the testing phase, but I do not face any problem in the training and validation phase. I get this error for the last line in the given code snippet. The code snippet looks like the one below, The “lab” is a tensor value and prints out the range in such a way, (N…
Amazon S3 boto3 how to iterate through objects in a bucket?
In a flask app, I was trying to iterate through objects in a S3 Bucket and trying to print the key/ filename but my_bucket.objects.all() returns only the first object in the bucket. It’s not returning the all the objects. The output is [001.pdf] instead of [001, 002, 003, 004, 005] Answer You are exitin…
Get In Focus Pixels of an Image
How to detect which pixels of an image are in focus compared to the blurry ones. Something like the ‘Focus Peaking’ feature lots of cameras have? The idea is to color the pixels that are in focus so that it assists the user while clicking a picture. Looking for an implementation through Python. An…
Manipulating 2D matrix using numpy boolean indexing takes a long time
I’ve generated a huge amount of random data like so: which is a 100,000 by 1000 matrix(!) I’m generating new matrix where for each row, each column is true if the mean of all the columns beforehand (minus the expectancy of bernoulli RV with p=0.25) is greater than equal some epsilon. like so: Afte…
Split CSV values on single row into individual rows
I have a Python script that outputs a text file with thousands of random filenames in a comma separated list, all on a single row. I want to take each value in the list and put it into its own row in a new CSV file. I’ve tried some variations of awk with no success. What’s the best way to
How to remove text and remain with interger values in python dataframe column
I have a dataframe with the column as follows; I want the output as follows: I have tried, No success so far. Answer This depends on what possible values can be in the quantity column. If there can never be a space in the numerical part (as in your example) you can use Series.str.partition: This can also be w…
SQLAlchemy engine.connect() fails for Snowflake using keypair authentication
My goal is to be able to write a pandas dataframe to Snowflake using the to_sql() function. I am able to successfuly query from Snowflake using read_sql() with no problems. It appears that to_sql() requires a slightly different configuration I can’t figure out. I’ve tried two different approaches …
How can a Google Cloud Python function access a private Python package
I am using Google Cloud Function using Python. Several other functions are in production. However, for this, I have additionally created a custom Python package that is available on github as a private repo. I need to install the package in the Google Function WHAT I HAVE DONE I run the Google Function in loc…
Save failed in Google Colab
I opened a number of tabs at the same time. I think that’s why Google Colab was not able to support the heavy load. The message stated: Save failed This file could not be saved. Please use the File menu to download the .ipynb and upload the notebook to make a copy that includes your recent changes. Is d…