Skip to content
Advertisement

string split with the value of another clumn PySpark

I have the following data frame

JavaScript

i want it to split path column with value of the item column in the same index

JavaScript

i’ve used this udf function

JavaScript

it worked very well But, i was wondering if there’s another way to do it with pyspark function because i can’t use in anyway the “org” to join with another dataframe or save it as a delta table it gives me this error

JavaScript

Advertisement

Answer

using .fillna("") to fill null value to “”. Like this:org = org.fillna("").withColumn('crb_url', split_udf('path','item')[0])

User contributions licensed under: CC BY-SA
10 People found this is helpful
Advertisement