Consider the following data frames in Python Pandas: DataframeA ColA ColB ColC 1 dog 439 1 cat 932 1 frog 932 2 dog 2122 2 cat 454 2 frog 773 3 dog 9223 3 cat 3012 3 frog 898 DataframeB ColD ColE 1 101 2 314 3 124 To note, ColB just repeats it’s string values as ColA iterates upwards.
Tag: dataframe
How to calculate average returns over separate consecutive ranges determined by another column in Python?
I currently have a Pandas DataFrame which contains a time series of asset prices and a column containing a “state”. There are three states -1, 0, 1 that occur at various points in the data. I am trying to find the average return on the asset in each of these states, ideally using a vectorised meth…
Python/Pandas:How to process a column of data like a dictionary
i have a csv lie this i would like to sum the values from column “PDCP.RxBytesUl”, PDCP.RxBytesUl = 5QI1+5QI2+5QI3+5QI4+5QI5+5QI6+5QI7+5QI8+5QI9 finally,the result is like this At first I wanted to convert this column into a dict(), but I found the format was not right, i have no idea, please help…
How to fill in missing values in Pandas dataframe according to pattern in column?
Suppose I have a dataframe with a column as follows: I want each row to be filled in with increments of 5 so that the final output would appear like: I’ve tried using np.arange and .reindex() but haven’t had much luck. I’m looking for an iterative approach instead of simply manually filling …
How to downcast numeric columns in Pandas?
How to optimize the data frame memory footprint and find the most optimal (minimal) data types dtypes for numeric columns. For example: Expected result: Answer You can use parameter downcast in to_numeric with selectig integers and floats columns by DataFrame.select_dtypes, it working from pandas 0.19+ like m…
How do I use sum and count functions together on different columns in my data frame function?
My data frame is the following: My current function is: It outputs the following: What is the best way to modify my function to give me the sum of the price as well? Example: I have looked at the .agg method but I’m lacking examples that use different columns. (I’m also not sure if that’s th…
How to multiply all numerical columns of a dataframe by a one-dimensional array?
I have a dataframe df of shape r x c, with 1 text column (with non-unique values) and the other columns all floats. I then have an array mult of size r. I would like each numerical column of the dataframe to be multiplied by the corresponding item of the array. Ie the desired output is that value_1[0] should …
Passing datetime64[ns] from pandas’ data frame as an argument to a function
I’m trying to create an additional column in a data frame to show the number of network days (excluding custom holidays) between two dates. I’m using a function to which I’m trying to pass dates from df’s columns as arguments, but I can’t make it work. Below is my code (I’m…
Extract parameterized data from string using python from AWS Lambda EVENT
This is the raw string I need to fetch value in DICT format, like I am unable to extract data as i wanted… i need suggestion how to extract data from the string? N.B: here string like “276035280975268320779229” changes every time i send request. Answer Here is an option using re.findall: Thi…
Drop duplicated rows based on multiple columns if other column(s) is NaNs in Pandas
Given a test dataset as follows: I would like to drop duplicated rows based on city and district, then drop rows if its quantity is NaN, but if city and district are not duplicated, then even if quantity is NaN, it’s not necessary to drop rows. Code based on link from here: Out: But I want to keep the l…