I am doing some topic modelling, and I am interested in showing how the average topic weight changes over time. The problem arises when I plot it using matplotlib (version 3.3.4). On the x-axis I would like to have the categorical month_year variable. The problem is that it is not ordered in a sensible way. I…
Tag: pandas
How do you slice a cross section in pandas or numpy?
I have the following data frame which can be copy/pasted and made to a data frame with: df = pd.read_clipboard() I would like to take a cross section from it, I want something like say: [1, 4, 9, 1, 10, 6, 4, 0, 4, 6, 10, 1, 9, 4, 1]) which is index df.loc[1, 0], df.loc[2, 1], df.loc[3, 2], df.loc[4,
ValueError: Shape of passed values is (4, 4), indices imply (1, 4)
Can someone explain to me what is the cause of this error? Answer The issue is the index=[‘col1’] argument. The data you’re passing to pd.DataFrame() is a list of 4 lists, where each list has 4 items, so there will be 4 rows and 4 columns. But, you’re setting the index (row labels) to …
Pandas appending dictionary values with iterrows row values
I have a dict of city names, each having an empty list as a value. I am trying to use df.iterrows() to append corresponding names to each dict key(city): Can somebody explain why the code above appends all possible ‘fullname’ values to each dict’s key instead of appending them to their respe…
Trouble subtracting two column values correctly/precisely in pandas dataframe in Python
I’m trying to create a new column in my pandas dataframe which will be the difference of two other columns, but the new column has values that are significantly different what what the differences between the values of the columns are. I have heard that ‘float’ values often don’t subtr…
Row wise operation in Pandas DataFrame
I have a Dataframe as I would like to have a lambda function in apply method to create a list of dictionary (including index item) as below If I use something like .apply(lambda x: <some operation>) here, x does not include the index rather the values. Cheers, DD Answer To expand Hans Bambel’s ans…
Pandas dataframe manipulation/re-sizing of a single-column count file
I have a file that looks like this: I want to read this into a pandas dataframe and re-shape it so that it looks like this: Is this possible? If so, how? Notes: it will not always be this size, so the solution needs to be size-independent. The input file will be max ~200gRNAs x 20genes. There will be gRNA_som…
Alternative to irregular nested np.where clauses
I’m struggling to simplify my irregular nested np.where clauses. Is there a way to make the code more readable? Answer Using np.select as suggested by @sammywemmy:
Python pandas decrease backfill until reach a certain number based on interval
I have the following dataframe called df, I want to do a backfill when a 1 appears in any column, and fill backwards until a number appears or failing that, backfill til a set number. So let’s say the set number to reduce o to is 0 and the decrement is 0.1, it should look like this, Can this be
Webscaping table data with drop down menu help in either Pandas, Beautiful Soup or Selenium
I am trying to scrape data from this website: https://www.shanghairanking.com/rankings/grsssd/2021 Initially pandas gets me out the gates and I can scrape the table but I am struggling with the drop down menus. I want to select the options next to the total score box which are PUB, CIT, etc. When I inspect th…