I have a log file that has large number of columns and I would like to merge timestamp columns into one as shown in the example below Note: Single ID can have multiple similar processes e.g. multiple A in an ID Trying with melt on pandas but it seem I am missing something here Answer IIUC, you can use melt
Tag: pandas
How can I add data to BigQuery without problems with rate limits? [closed]
Closed. This question needs to be more focused. It is not currently accepting answers. Want to improve this question? Update the question so it focuses on one problem only by editing this post. Closed 8 months ago. Improve this question I currently have a system in which I want to send data from that system v…
Format pandas dataframe output into a text file as a table (formatted and aligned to the max length of the data or header (which ever is longer))
I have the above data frame and would like to save the output in a file as a pipe delimited data like below. So far I have tried pd.to_csv and pd.to_string(), both outputs the data in tabular format however, the data is not aligning to the max length of the column header or the data. to_string() to_csv() Answ…
how to read a file from a local directory from python code in a docker container?
I have tried to containerize my python code , this is working, building the docker image, running the container and invoking the python code in my app.py file. next, i want to be able to upload the sample.txt file that my python code reads from the root of the application to process some data, sample.txt in t…
How to cross-reference data in Pandas dataframes?
I’m working with data that has two separate IDs per item. When we pull data from most sources, we get a PLU/SKU—however, in one of our sources, we get an item number from our on-prem point-of-sale system. To solve this by hand, we have a master list that contains both the PLU and item number for each it…
How can I merge aggregate two dataframes in Pandas while subtracting column values?
I’m working on a rudimentary inventory system and am having trouble finding a solution to this obstacle. I’ve got two Pandas dataframes, both sharing two columns: PLU and QTY. PLU acts as an item identifier, and QTY is the quantity of the item in one dataframe, while being the quantity sold in ano…
python find the nearest nonzero element in df column
I have df: I want to write a function to fill 0 values.I want for each id(for each group) , when the value in number column is zero, to search the closet non zero value in the column and return the value. for example to id 1 to fill the second and third-row with 2. If I dont have such
Find words in array and get their indexes in Dataframe in Pandas
I have a dataframe : I need to find words from column token_1 in token_2 and get their indixes in an array. Then get a list of indexes for each line, i expected this: Answer Use list comprehension with enumerate for indices:
Visualize how multiple categorical values differ across rows and columns in a dataframe
I have the following DataFrame where each column represents a categorization algorithm for the items in the index (a,b, …) I would like to reorder the category names in each column so that I can better assess whether the index items are being categorised similarly across columns. Is there a way to visualise h…
Find value smaller but closest to current value
I have a very large pandas dataframe that contains two columns, column A and column B. For each value in column A, I would like to find the largest value in column B that is less than the corresponding value in column A. Note that each value in column B can be mapped to many values in column A. Here’s