I am currently trying to use Pandas to sort through Can-Bus data however when I try to make a DataFrame with 2 similar text files, I get two completely different DataFrames.
When I make a data frame with the txt file “CANDUMP With Codes.txt” I get the following DataFrame:
0 ID B0 B1 B2 B3 B4 B5 B6 B7 0 1.656033e+09 0D0 4E FD 00 00 00 00 FF FF 1 1.656033e+09 0D1 00 00 00 00 NaN NaN NaN NaN 2 1.656033e+09 0D2 00 00 FF FF 00 00 00 00 3 1.656033e+09 0D3 00 06 C0 0F 48 00 72 NaN 4 1.656033e+09 0D4 00 00 00 00 00 00 00 00 5 1.656033e+09 018 02 91 70 00 00 00 00 23 6 1.656033e+09 140 00 0F 89 42 00 00 04 01 7 1.656033e+09 141 84 26 5C 27 87 82 A7 00 8 1.656033e+09 142 84 26 00 00 00 00 00 00 9 1.656033e+09 144 C0 00 51 0A 05 A4 A0 00 10 1.656033e+09 156 00 00 00 00 00 00 00 00 11 1.656033e+09 018 02 91 70 02 00 00 00 25 12 1.656033e+09 140 00 00 88 42 00 00 04 01 13 1.656033e+09 141 84 26 5C 27 87 82 A7 00 14 1.656033e+09 142 84 26 00 00 00 00 00 00 15 1.656033e+09 360 B9 00 7C 82 28 80 01 00 16 1.656033e+09 361 00 29 00 D9 04 70 74 75 17 1.656033e+09 152 F0 E4 00 00 00 02 08 00 18 1.656033e+09 0D0 4E FD 00 00 00 00 00 FF 19 1.656033e+09 0D1 00 00 00 00 NaN NaN NaN NaN 20 1.656033e+09 0D2 00 00 FF FF 00 00 00 00 21 1.656033e+09 0D3 00 06 C0 0F 48 00 73 NaN 22 1.656033e+09 0D4 00 00 00 00 00 00 00 00 23 1.656033e+09 018 02 91 70 04 00 00 00 27 24 1.656033e+09 140 00 01 88 42 00 00 04 01 25 1.656033e+09 141 84 26 5C 27 87 82 A7 00 26 1.656033e+09 142 84 26 00 00 00 00 00 00 27 1.656033e+09 144 C0 00 51 0A 05 A4 A0 00 28 1.656033e+09 156 00 00 00 00 00 00 00 00 29 1.656033e+09 6E1 A1 80 00 00 00 60 00 00 30 1.656033e+09 018 02 91 70 06 00 00 00 29 31 1.656033e+09 140 00 02 88 42 00 00 04 01 32 1.656033e+09 141 84 26 5C 27 87 82 A7 00 33 1.656033e+09 142 84 26 00 00 00 00 00 00 34 1.656033e+09 152 F0 F4 00 00 00 02 08 00 35 1.656033e+09 0D0 4E FD 00 00 00 00 FF FF 36 1.656033e+09 0D1 00 00 00 00 NaN NaN NaN NaN 37 1.656033e+09 0D2 00 00 FF FF 00 00 00 00 38 1.656033e+09 0D3 00 06 C0 0F 48 00 74 NaN 39 1.656033e+09 0D4 00 00 00 00 00 00 00 00 40 1.656033e+09 4C3 03 00 00 00 00 00 00 00 41 1.656033e+09 018 02 91 70 08 00 00 00 2B 42 1.656033e+09 140 00 03 88 42 00 00 04 01 43 1.656033e+09 141 84 26 5C 27 87 82 A7 00 44 1.656033e+09 142 84 26 00 00 00 00 00 00 45 1.656033e+09 144 C0 00 51 0A 05 A4 A0 00 46 1.656033e+09 156 00 00 00 00 00 00 00 00 47 1.656033e+09 370 00 00 00 00 00 05 00 00 48 1.656033e+09 018 02 91 70 0A 00 00 00 2D 49 1.656033e+09 140 00 04 8B 42 00 00 04 01 50 1.656033e+09 282 88 94 FA 60 61 03 30 00 51 1.656033e+09 152 F0 04 00 00 00 02 08 00 52 1.656033e+09 0D0 4E FD 00 00 00 00 FF FF 53 1.656033e+09 0D1 00 00 00 00 NaN NaN NaN NaN 54 1.656033e+09 0D2 00 00 FF FF 00 00 00 00 55 1.656033e+09 0D3 00 06 C0 0F 48 00 75 NaN 56 1.656033e+09 0D4 00 00 00 00 00 00 00 00 57 1.656033e+09 018 02 91 70 0C 00 00 00 2F 58 1.656033e+09 140 00 05 8B 42 00 00 04 01 59 1.656033e+09 141 84 26 5C 27 88 82 A7 00 60 1.656033e+09 142 84 26 00 00 00 00 00 00 61 1.656033e+09 144 C0 00 51 0A 05 A4 A0 00 62 1.656033e+09 360 BA 00 7C 82 28 80 01 00 63 1.656033e+09 361 00 29 00 D9 05 E0 7C 75 64 1.656033e+09 156 00 00 00 00 00 00 00 00 65 1.656033e+09 440 42 02 00 00 00 00 00 00 66 1.656033e+09 372 00 70 00 00 00 00 00 00 67 1.656033e+09 018 02 91 70 0E 00 00 00 31 68 1.656033e+09 140 00 06 8B 42 00 00 04 01 69 1.656033e+09 141 84 26 5C 27 88 82 A7 00 70 1.656033e+09 142 84 26 00 00 00 00 00 00 71 1.656033e+09 4C1 01 00 00 00 00 00 00 00 72 1.656033e+09 152 F0 14 00 00 00 02 08 00 73 1.656033e+09 0D0 4E FD 00 00 00 00 00 FF 74 1.656033e+09 0D1 00 00 00 00 NaN NaN NaN NaN 75 1.656033e+09 0D2 00 00 FF FF 00 00 00 00 76 1.656033e+09 0D3 00 06 C0 0F 48 00 76 NaN 77 1.656033e+09 0D4 00 00 00 00 00 00 00 00 78 1.656033e+09 018 02 91 70 00 00 00 00 23 79 1.656033e+09 140 00 07 8B 42 00 00 04 01 80 1.656033e+09 141 84 26 5C 27 88 82 A7 00 81 1.656033e+09 142 84 26 00 00 00 00 00 00 82 1.656033e+09 144 C0 00 51 0A 05 A4 A0 00 83 1.656033e+09 156 00 00 00 00 00 00 00 00 84 1.656033e+09 018 02 91 70 02 00 00 00 25 85 1.656033e+09 140 00 08 8B 42 00 00 04 01 86 1.656033e+09 141 84 26 5C 27 88 82 A7 00 87 1.656033e+09 142 84 26 00 00 00 00 00 00 88 1.656033e+09 370 00 00 00 00 00 06 00 00 89 1.656033e+09 152 F0 24 00 00 00 02 08 00 90 1.656033e+09 0D0 4E FD 00 00 00 00 FF FF 91 1.656033e+09 0D1 00 00 00 00 NaN NaN NaN NaN 92 1.656033e+09 0D2 00 00 FF FF 00 00 00 00 93 1.656033e+09 0D3 00 06 C0 0F 48 00 77 NaN 94 1.656033e+09 0D4 00 00 00 00 00 00 00 00 95 1.656033e+09 018 02 91 70 04 00 00 00 27 96 1.656033e+09 140 00 09 89 42 00 00 04 01 97 1.656033e+09 141 84 26 5C 27 88 82 A7 00 98 1.656033e+09 142 84 26 00 00 00 00 00 00 99 1.656033e+09 144 C0 00 51 0A 05 A4 A0 00 100 1.656033e+09 156 00 00 00 00 00 00 00 00
When I make a DataFrame with the txt file “ID 00d0.txt” I get the following DataFrame:
0 ID B0 B1 B2 B3 B4 B5 B6 B7 0 1.656033e+09 0d0 A0 0 0 0 0 0 ff 1 1 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 2 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 3 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 4 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 5 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 6 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 7 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 8 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 9 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 10 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 11 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 12 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 13 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 14 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 15 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 16 1.656033e+09 0d0 30 0 1 0 0 0 00 1 17 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 18 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 19 1.656033e+09 0d0 30 0 1 0 0 0 00 1 20 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 21 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 22 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 23 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 24 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 25 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 26 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 27 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 28 1.656033e+09 0d0 30 0 1 0 0 0 ff 2 29 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 30 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 31 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 32 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 33 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 34 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 35 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 36 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 37 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 38 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 39 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 40 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 41 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 42 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 43 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 44 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 45 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 46 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 47 1.656033e+09 0d0 30 0 0 0 0 0 00 2 48 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 49 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 50 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 51 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 52 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 53 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 54 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 55 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 56 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 57 1.656033e+09 0d0 30 0 0 0 0 0 00 1 58 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 59 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 60 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 61 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 62 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 63 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 64 1.656033e+09 0d0 30 0 1 0 0 0 ff 1 65 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 66 1.656033e+09 0d0 30 0 0 0 0 0 00 1 67 1.656033e+09 0d0 30 0 0 0 0 0 ff 1 68 1.656033e+09 0d0 2f 0 0 0 0 0 ff 1 69 1.656033e+09 0d0 2f 0 0 0 0 0 ff 1 70 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 71 1.656033e+09 0d0 2f 0 0 0 0 0 ff 1 72 1.656033e+09 0d0 2f 0 0 0 0 0 ff 2 73 1.656033e+09 0d0 2f 0 0 0 0 0 ff 1 74 1.656033e+09 0d0 2f 0 1 0 0 0 fe 1 75 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 76 1.656033e+09 0d0 2f 0 1 0 0 0 00 1 77 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 78 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 79 1.656033e+09 0d0 2f 0 0 0 0 0 ff 1 80 1.656033e+09 0d0 2f 0 0 0 0 0 ff 1 81 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 82 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 83 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 84 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 85 1.656033e+09 0d0 2f 0 1 0 0 0 00 2 86 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 87 1.656033e+09 0d0 2f 0 1 0 0 0 fe 1 88 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 89 1.656033e+09 0d0 2f 0 0 0 0 0 00 1 90 1.656033e+09 0d0 2f 0 0 0 0 0 ff 1 91 1.656033e+09 0d0 2f 0 0 0 0 0 ff 1 92 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 93 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 94 1.656033e+09 0d0 2f 0 0 0 0 0 ff 2 95 1.656033e+09 0d0 2f 0 0 0 0 0 ff 1 96 1.656033e+09 0d0 2f 0 0 0 0 0 ff 1 97 1.656033e+09 0d0 2f 0 0 0 0 0 fe 1 98 1.656033e+09 0d0 2f 0 0 0 0 0 ff 1 99 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1 100 1.656033e+09 0d0 2f 0 1 0 0 0 ff 1
When I try to add some values together from ID 00d0 I get errors of mixing str with int values however with the “CANDUMP With Codes” file this does not happen. The problem is that I need ID 00d0 to be stored using hex (str) like the CANDUMP file.
Down below is the sample code I used to get the DataFrames as well as the content of the text files (I dont know if you can add txt files to a post so I put it in code below). If anyone has any idea why this is happening and has a potential solution, I am all for it.
Thank you all for your time.
Code I am using:
import pandas as pd
import time
startTime = time.time()
pd.set_option('display.max_rows', None)
df = pd.read_table("ID 00d0.txt", header=None, delim_whitespace=True)
df.columns = ['0','ID',"B0","B1","B2","B3","B4","B5","B6","B7"]
df2 = pd.read_table("CANDUMP With Codes.txt", header=None, delim_whitespace=True)
df2.columns = ['0','ID',"B0","B1","B2","B3","B4","B5","B6","B7"]
print(df2)
executionTime = round((time.time() - startTime),10)
print('Execution time in seconds: ' + str(executionTime))
CANDUMP With Codes.txt:
1656033437.571007 0D0 4E FD 00 00 00 00 FF FF 1656033437.571007 0D1 00 00 00 00 1656033437.571007 0D2 00 00 FF FF 00 00 00 00 1656033437.571007 0D3 00 06 C0 0F 48 00 72 1656033437.571007 0D4 00 00 00 00 00 00 00 00 1656033437.571007 018 02 91 70 00 00 00 00 23 1656033437.571007 140 00 0F 89 42 00 00 04 01 1656033437.571007 141 84 26 5C 27 87 82 A7 00 1656033437.571007 142 84 26 00 00 00 00 00 00 1656033437.571007 144 C0 00 51 0A 05 A4 A0 00 1656033437.571007 156 00 00 00 00 00 00 00 00 1656033437.571007 018 02 91 70 02 00 00 00 25 1656033437.571007 140 00 00 88 42 00 00 04 01 1656033437.571007 141 84 26 5C 27 87 82 A7 00 1656033437.571007 142 84 26 00 00 00 00 00 00 1656033437.571007 360 B9 00 7C 82 28 80 01 00 1656033437.571007 361 00 29 00 D9 04 70 74 75 1656033437.571007 152 F0 E4 00 00 00 02 08 00 1656033437.571007 0D0 4E FD 00 00 00 00 00 FF 1656033437.571007 0D1 00 00 00 00 1656033437.571007 0D2 00 00 FF FF 00 00 00 00 1656033437.571007 0D3 00 06 C0 0F 48 00 73 1656033437.571007 0D4 00 00 00 00 00 00 00 00 1656033437.571007 018 02 91 70 04 00 00 00 27 1656033437.571007 140 00 01 88 42 00 00 04 01 1656033437.571007 141 84 26 5C 27 87 82 A7 00 1656033437.571007 142 84 26 00 00 00 00 00 00 1656033437.571007 144 C0 00 51 0A 05 A4 A0 00 1656033437.571007 156 00 00 00 00 00 00 00 00 1656033437.571007 6E1 A1 80 00 00 00 60 00 00 1656033437.571007 018 02 91 70 06 00 00 00 29 1656033437.571007 140 00 02 88 42 00 00 04 01 1656033437.571007 141 84 26 5C 27 87 82 A7 00 1656033437.571007 142 84 26 00 00 00 00 00 00 1656033437.571007 152 F0 F4 00 00 00 02 08 00 1656033437.571007 0D0 4E FD 00 00 00 00 FF FF 1656033437.571007 0D1 00 00 00 00 1656033437.571007 0D2 00 00 FF FF 00 00 00 00 1656033437.571007 0D3 00 06 C0 0F 48 00 74 1656033437.571007 0D4 00 00 00 00 00 00 00 00 1656033437.571007 4C3 03 00 00 00 00 00 00 00 1656033437.571007 018 02 91 70 08 00 00 00 2B 1656033437.571007 140 00 03 88 42 00 00 04 01 1656033437.571007 141 84 26 5C 27 87 82 A7 00 1656033437.571007 142 84 26 00 00 00 00 00 00 1656033437.571007 144 C0 00 51 0A 05 A4 A0 00 1656033437.571007 156 00 00 00 00 00 00 00 00 1656033437.571007 370 00 00 00 00 00 05 00 00 1656033437.571007 018 02 91 70 0A 00 00 00 2D 1656033437.571007 140 00 04 8B 42 00 00 04 01 1656033437.571007 282 88 94 FA 60 61 03 30 00 1656033437.571007 152 F0 04 00 00 00 02 08 00 1656033437.571007 0D0 4E FD 00 00 00 00 FF FF 1656033437.571007 0D1 00 00 00 00 1656033437.571007 0D2 00 00 FF FF 00 00 00 00 1656033437.571007 0D3 00 06 C0 0F 48 00 75 1656033437.571007 0D4 00 00 00 00 00 00 00 00 1656033437.571007 018 02 91 70 0C 00 00 00 2F 1656033437.571007 140 00 05 8B 42 00 00 04 01 1656033437.571007 141 84 26 5C 27 88 82 A7 00 1656033437.571007 142 84 26 00 00 00 00 00 00 1656033437.571007 144 C0 00 51 0A 05 A4 A0 00 1656033437.571007 360 BA 00 7C 82 28 80 01 00 1656033437.571007 361 00 29 00 D9 05 E0 7C 75 1656033437.571007 156 00 00 00 00 00 00 00 00 1656033437.571007 440 42 02 00 00 00 00 00 00 1656033437.571007 372 00 70 00 00 00 00 00 00 1656033437.571007 018 02 91 70 0E 00 00 00 31 1656033437.571007 140 00 06 8B 42 00 00 04 01 1656033437.571007 141 84 26 5C 27 88 82 A7 00 1656033437.571007 142 84 26 00 00 00 00 00 00 1656033437.571007 4C1 01 00 00 00 00 00 00 00 1656033437.571007 152 F0 14 00 00 00 02 08 00 1656033437.571007 0D0 4E FD 00 00 00 00 00 FF 1656033437.571007 0D1 00 00 00 00 1656033437.571007 0D2 00 00 FF FF 00 00 00 00 1656033437.571007 0D3 00 06 C0 0F 48 00 76 1656033437.571007 0D4 00 00 00 00 00 00 00 00 1656033437.571007 018 02 91 70 00 00 00 00 23 1656033437.571007 140 00 07 8B 42 00 00 04 01 1656033437.571007 141 84 26 5C 27 88 82 A7 00 1656033437.571007 142 84 26 00 00 00 00 00 00 1656033437.571007 144 C0 00 51 0A 05 A4 A0 00 1656033437.571007 156 00 00 00 00 00 00 00 00 1656033437.571007 018 02 91 70 02 00 00 00 25 1656033437.571007 140 00 08 8B 42 00 00 04 01 1656033437.571007 141 84 26 5C 27 88 82 A7 00 1656033437.571007 142 84 26 00 00 00 00 00 00 1656033437.571007 370 00 00 00 00 00 06 00 00 1656033437.571007 152 F0 24 00 00 00 02 08 00 1656033437.571007 0D0 4E FD 00 00 00 00 FF FF 1656033437.571007 0D1 00 00 00 00 1656033437.571007 0D2 00 00 FF FF 00 00 00 00 1656033437.571007 0D3 00 06 C0 0F 48 00 77 1656033437.571007 0D4 00 00 00 00 00 00 00 00 1656033437.571007 018 02 91 70 04 00 00 00 27 1656033437.571007 140 00 09 89 42 00 00 04 01 1656033437.571007 141 84 26 5C 27 88 82 A7 00 1656033437.571007 142 84 26 00 00 00 00 00 00 1656033437.571007 144 C0 00 51 0A 05 A4 A0 00 1656033437.571007 156 00 00 00 00 00 00 00 00
ID 00d0.txt:
1656033437.571007 00d0 A0 00 00 00 00 00 ff 01 1656033437.590747 00d0 30 00 00 00 00 00 ff 01 1656033437.610978 00d0 30 00 00 00 00 00 ff 01 1656033437.630766 00d0 30 00 00 00 00 00 ff 01 1656033437.650933 00d0 30 00 00 00 00 00 ff 01 1656033437.670918 00d0 30 00 00 00 00 00 ff 01 1656033437.690742 00d0 30 00 00 00 00 00 ff 01 1656033437.710797 00d0 30 00 00 00 00 00 ff 01 1656033437.730938 00d0 30 00 00 00 00 00 ff 01 1656033437.750732 00d0 30 00 00 00 00 00 ff 01 1656033437.770774 00d0 30 00 00 00 00 00 ff 01 1656033437.790819 00d0 30 00 00 00 00 00 ff 01 1656033437.810714 00d0 30 00 00 00 00 00 ff 01 1656033437.830694 00d0 30 00 00 00 00 00 ff 01 1656033437.850853 00d0 30 00 00 00 00 00 ff 01 1656033437.870689 00d0 30 00 00 00 00 00 ff 01 1656033437.890834 00d0 30 00 01 00 00 00 00 01 1656033437.910732 00d0 30 00 01 00 00 00 ff 01 1656033437.930933 00d0 30 00 01 00 00 00 ff 01 1656033437.950721 00d0 30 00 01 00 00 00 00 01 1656033437.970946 00d0 30 00 01 00 00 00 ff 01 1656033437.990844 00d0 30 00 01 00 00 00 ff 01 1656033438.010802 00d0 30 00 01 00 00 00 ff 01 1656033438.030679 00d0 30 00 01 00 00 00 ff 01 1656033438.050847 00d0 30 00 01 00 00 00 ff 01 1656033438.070932 00d0 30 00 01 00 00 00 ff 01 1656033438.090849 00d0 30 00 01 00 00 00 ff 01 1656033438.110787 00d0 30 00 01 00 00 00 ff 01 1656033438.130893 00d0 30 00 01 00 00 00 ff 02 1656033438.151006 00d0 30 00 01 00 00 00 ff 01 1656033438.170744 00d0 30 00 01 00 00 00 ff 01 1656033438.190754 00d0 30 00 01 00 00 00 ff 01 1656033438.210979 00d0 30 00 01 00 00 00 ff 01 1656033438.230791 00d0 30 00 00 00 00 00 ff 01 1656033438.250794 00d0 30 00 00 00 00 00 ff 01 1656033438.270932 00d0 30 00 00 00 00 00 ff 01 1656033438.290712 00d0 30 00 00 00 00 00 ff 01 1656033438.310696 00d0 30 00 00 00 00 00 ff 01 1656033438.330892 00d0 30 00 00 00 00 00 ff 01 1656033438.35081 00d0 30 00 00 00 00 00 ff 01 1656033438.370926 00d0 30 00 00 00 00 00 ff 01 1656033438.390713 00d0 30 00 00 00 00 00 ff 01 1656033438.410782 00d0 30 00 00 00 00 00 ff 01 1656033438.430785 00d0 30 00 00 00 00 00 ff 01 1656033438.450719 00d0 30 00 01 00 00 00 ff 01 1656033438.470731 00d0 30 00 01 00 00 00 ff 01 1656033438.490977 00d0 30 00 00 00 00 00 ff 01 1656033438.510828 00d0 30 00 00 00 00 00 00 02 1656033438.530749 00d0 30 00 00 00 00 00 ff 01 1656033438.55073 00d0 30 00 00 00 00 00 ff 01 1656033438.570944 00d0 30 00 00 00 00 00 ff 01 1656033438.590847 00d0 30 00 00 00 00 00 ff 01 1656033438.610798 00d0 30 00 00 00 00 00 ff 01 1656033438.630972 00d0 30 00 00 00 00 00 ff 01 1656033438.650738 00d0 30 00 00 00 00 00 ff 01 1656033438.670883 00d0 30 00 01 00 00 00 ff 01 1656033438.690741 00d0 30 00 00 00 00 00 ff 01 1656033438.710854 00d0 30 00 00 00 00 00 00 01 1656033438.730805 00d0 30 00 00 00 00 00 ff 01 1656033438.750711 00d0 30 00 00 00 00 00 ff 01 1656033438.770848 00d0 30 00 00 00 00 00 ff 01 1656033438.790796 00d0 30 00 00 00 00 00 ff 01 1656033438.81073 00d0 30 00 00 00 00 00 ff 01 1656033438.830729 00d0 30 00 01 00 00 00 ff 01 1656033438.850798 00d0 30 00 01 00 00 00 ff 01 1656033438.870746 00d0 30 00 00 00 00 00 ff 01 1656033438.890738 00d0 30 00 00 00 00 00 00 01 1656033438.910826 00d0 30 00 00 00 00 00 ff 01 1656033438.930735 00d0 2f 00 00 00 00 00 ff 01 1656033438.950811 00d0 2f 00 00 00 00 00 ff 01 1656033438.970722 00d0 2f 00 01 00 00 00 ff 01 1656033438.990992 00d0 2f 00 00 00 00 00 ff 01 1656033439.010762 00d0 2f 00 00 00 00 00 ff 02 1656033439.03074 00d0 2f 00 00 00 00 00 ff 01 1656033439.050974 00d0 2f 00 01 00 00 00 fe 01 1656033439.070785 00d0 2f 00 01 00 00 00 ff 01 1656033439.090808 00d0 2f 00 01 00 00 00 00 01 1656033439.110784 00d0 2f 00 01 00 00 00 ff 01 1656033439.130923 00d0 2f 00 01 00 00 00 ff 01 1656033439.150792 00d0 2f 00 00 00 00 00 ff 01 1656033439.170964 00d0 2f 00 00 00 00 00 ff 01 1656033439.190788 00d0 2f 00 01 00 00 00 ff 01 1656033439.210718 00d0 2f 00 01 00 00 00 ff 01 1656033439.230836 00d0 2f 00 01 00 00 00 ff 01 1656033439.250775 00d0 2f 00 01 00 00 00 ff 01 1656033439.270921 00d0 2f 00 01 00 00 00 00 02 1656033439.290832 00d0 2f 00 01 00 00 00 ff 01 1656033439.310932 00d0 2f 00 01 00 00 00 fe 01 1656033439.330775 00d0 2f 00 01 00 00 00 ff 01 1656033439.350807 00d0 2f 00 00 00 00 00 00 01 1656033439.37097 00d0 2f 00 00 00 00 00 ff 01 1656033439.390729 00d0 2f 00 00 00 00 00 ff 01 1656033439.411024 00d0 2f 00 01 00 00 00 ff 01 1656033439.430715 00d0 2f 00 01 00 00 00 ff 01 1656033439.450694 00d0 2f 00 00 00 00 00 ff 02 1656033439.47096 00d0 2f 00 00 00 00 00 ff 01 1656033439.490835 00d0 2f 00 00 00 00 00 ff 01 1656033439.51084 00d0 2f 00 00 00 00 00 fe 01 1656033439.530714 00d0 2f 00 00 00 00 00 ff 01 1656033439.550823 00d0 2f 00 01 00 00 00 ff 01 1656033439.570697 00d0 2f 00 01 00 00 00 ff 01
Advertisement
Answer
Why does this happen?
If you don’t tell pandas in which format the data is, it will guess this for you. Since ID 00d0.txt contains only 00 in some columns, the type is ambigous and pandas chose to parse it as an int. You can see all datatypes for each dataframe by calling
df.info()
and looking at the Dtypes column.
What is the fix?
An easy fix would be to read in everything as a string. This is done by adding the keyword argument dtype=str to your read_table call like so:
df = pd.read_table("file.txt", header=None, delim_whitespace=True, dtype=str)
This way you can merge and combine dataframes as you please. Later, however, you may want to do some calculations on the columns, you could then convert them using an arbitrary function. For example to covert the hex strings into integers:
df["B0"] = df["B0"].apply(lambda x: int(x, 16))
Hope that helps :)