I merged 3 different CSV(D1,D2,D3) Netflow datasets and created one big dataset(df), and applied KMeans clustering to this dataset. To merge them I did not use pd.concat because of memory error and solved with Linux terminal. All these datasets contain the same column names, they have 12 columns(all numerical values) Example expected result: cluster_0 has xxxx numbers of same rows
Tag: pandas
Create Pandas DataFrame from 2 tuple lists with common first elements
I currently have 2 lists of tuples, both of which have the same information on the first element of the tuple. I’m trying to see if there is a way to “Join” these two tuple lists in a dataframe based on their common elements. Something like an SQL JOIN on a common column. Lists are something like this: and what
How can I combine different dataframes into one csv in Python?
I have 2 dataframes with different columns. And I want to combine those into 1 csv file. Both headers should be included and there shouldn’t be empty value if columns aren’t matched. I tried to use pd.concat, but I need the result to be like below: Answer You can do this using Pandas to_csv and setting the mode parameter to
How to find if if a particular column has zero value in a dataframe?
Trust you all are doing well! I have this dataframe that contains 0 and float numbers in column (‘BP_MOVE’) , there could be two conditions Zero in any row of column (‘BP_MOVE’) Zero in each row of column (‘BP_MOVE’) Below is what i have tried, the first python statement covers the second case very well where each row has 0
Pandas chart using plotly.graph_objects DO NOT ALLOW FOR ‘c’ or ‘color’ ATTRIBUTE
I am adding line charts to an OHLV plotly, but I do not manage to color them. The above code fail when I use ‘color’ as a ‘figg.add_scatter’ attribute : Error given is: WHEN I do not specidy the color of the the scatter lines, the program works fine. Bellow is a data sample: I would like to pick the
DataFrame has two features how to add a row to split them
I have a DataFrame that contains a column called feature that can have more than one of them as illustrated in the image below row 3 & 4. How do a add a row to the DataFrame that splits the two features: so for row 3 as an example having: and row 4: so the idea is to add a
Trying to get ‘QS’ frequency in pandas for a datetime64[ns] gives attribute error
I am working with an external data source and I am trying to get Quaterstart(QS) frequency for a particular data field. I am providing a dummy data and code below. I am gettinng the following error when I run this Can someone please help me understand what’s happening here? ps: The data given here is dummy data and not the
Linear regression prediction based on group of data in test set
I have a simple dataset which looks like this: I created a simple LR model to train and predict the target variable “sales”. And I used MAE to evaluate the model My code works well, but what I want to do is to predict the sales in the X_test grouped by hour of the day. In the above dataset example,
How to set merge and normalize multple dataframes for pd.merge_as_of
I am trying to merge multiple dataframes using pd.merge_asof. They all contain 2 columns with datetime as index column and a variable column with floating values. They are not balanced in their indexes and times so I have to normalize the values. I can succesfully merge the dfs and normalize the values like this using Now what I want to
Error tokenizing data. C error: Expected x fields in line 5, saw x
I keep getting this error. I don’t even know how to identify the row that is in error as the data I am requesting is jumbled. I can’t provide a URL to the API but I will provide a sample of the first few lines of data. My code: Error: Data from API: Answer Since you don’t specify a separator