I am using SQLAlchemy to read data from a SQL Server database and then turning the table data to a csv file to later hand off. However, I noticed when there is a 1 or 0 in a SQL Server table field, the csv output has True or False instead. I know to Python, it’s still a number since True
Tag: dtype
Is there a way to get the count of every element in lists stored as rows in a data frame?
Hi, I’m using pandas to display and analyze a csv file, some columns were ‘object dtype’ and were displayed as lists, I used ‘literal_eval’ to convert the rows of a column named ‘sdgs’ to lists, my problem is how to use ‘groupby’ or any another way to display the count of every element stored at this lists uniquely, especially since
Why tensor size was not changed?
I made the toy CNN model. Then, I had checked model.summary via this code And I was able to get the following results: I want to reduce model size cuz i wanna increase the batch size. So, I had changed torch.float32 -> torch.float16 via NVIDIA/apex As a result, torch.dtype was changed torch.float16 from torch.float32. But, Param size (MB): 35.19 was
Python object array of empty arrays
I am attempting to create a numpy array of empty arrays without using loops. Using loops, I can use a simplified operation like Or a marginally more sophisticated approach based on np.nditer: I can’t seem to find an indexing expression that will allow me to make this assignment in a vectorized manner. I’ve tried the following: Even np.nditer does not
Converting string in a Pandas data frame to float
I have the following data frame: In order to calculate with the second column named “Marktwert”, I have to convert the string as a float, the sting has German format, that means the decimal point is a comma and the thousands separator is a dot. The number 217.803,37 has the datatype object. If I try to convert using the code
match dtypes of one df to another with different number of columns
I have a dataframe that has 3 columns and looks like this: The other dataframe looks like this: I need to match the data types of one df to another. Because I have one additional column in df_1 I got an error. My code looks like this: I got an error: KeyError: ‘profitable’ What would be a workaround here? I
How to downcast numeric columns in Pandas?
How to optimize the data frame memory footprint and find the most optimal (minimal) data types dtypes for numeric columns. For example: Expected result: Answer You can use parameter downcast in to_numeric with selectig integers and floats columns by DataFrame.select_dtypes, it working from pandas 0.19+ like mentioned @anurag, thank you:
LightGBM does not accept the dtypes of my data
I’m trying to use LGBMClassifier and for some reason, he does not accept the types of my data (all features are not accepted, I tested it). When we look at my data we can clearly see that all dtypes are either category, float or int (pd.DataFrame.info()) When I eventually try to train my LGBMClassifier I get the follwoing Error: Has