I have below 2 pyspark dataframe df1 and df2 :
JavaScript
x
11
11
1
df1
2
product 04-01 04-02 04-03 04-05 04-06
3
cycle 12 24 25 17 39
4
bike 42 15 4 94 03
5
bycyle 111 23 12 04 95
6
7
8
df2
9
04-01 04-02 04-03 04-05 04-06
10
1 2 3 4 5
11
I want to multiply df1 each row with the same column of df2 row. Final output be like
JavaScript
1
6
1
result
2
product 04-01 04-02 04-03 04-05 04-06
3
cycle 12 48 75 68 195
4
bike 42 30 12 376 15
5
bycyle 111 46 36 16 475
6
Advertisement
Answer
You can do a cross join and multiply the columns using a list comprehension:
JavaScript
1
14
14
1
result = df1.crossJoin(df2).select(
2
'product',
3
*[(df1[c]*df2[c]).alias(c) for c in df1.columns[1:]]
4
)
5
6
result.show()
7
+-------+-----+-----+-----+-----+-----+
8
|product|04-01|04-02|04-03|04-05|04-06|
9
+-------+-----+-----+-----+-----+-----+
10
| cycle| 12| 48| 75| 68| 195|
11
| bike| 42| 30| 12| 376| 15|
12
| bycyle| 111| 46| 36| 16| 475|
13
+-------+-----+-----+-----+-----+-----+
14