Skip to content
Advertisement

How to reshape data delimited with whitespaces but with headers that can’t be delimited by white space

I’ve a simulation result for an antenna result as it can be found in here

You can see that I need to reshape this data so three things happen:

  • The headers are kept on the format Var(extraInfo)[unit] where the parenthesis can be sometimes optional as seen from the first column and third column
  • The second row is removed (the ---)
  • The data then is properly assigned to each of the header columns, which can be done easily through a `delim_whitespace=True)

The first and second topic seems the confusing to me (where to start at least).

And I also had a workaround so far that requires me to type this all every time (I’d like to make this more automatize hence the post). I basically ignore the two first rows and then manually enter the variables name

import pandas as pd

df = pd.read_csv('Results.txt', skiprows=2, delim_whitespace=True, lineterminator='n',header  =None)
df = df.drop(8, axis=1)
df.rename(columns={0: 'V1', 1: 'V2', 2: 'V3', 3: 'V4', 4: 'V5',5: 'V6', 6: 'V7', 7: 'V8'}, inplace=True)

Advertisement

Answer

I’ve since worked on this code. You can retrieve an example of the dataset here. Paste on your notepad and save as a .txt to reproduce. As I wanted to have freedom to use the different columns I decided to move from pandas towards numpy which made things a bit easier to play with (specially since all the processing ends up happening with numpy arrays).

import numpy as np

myArray = np.loadtxt('myFile.txt', skiprows=2)

If you look into the created array you can see you can select the columns as I wished for initially by doing:

myArray[:,0]
Out[99]: array([-180., -179., -178., ...,  177.,  178.,  179.])
Advertisement