I have two Airflow tasks that I want to communicate. The SSHOperator returns the last line printed, in this case, “remote_IP”. However, the SSHOperator’s return value is encoded using UTF-8.
JavaScript
x
11
11
1
Read_remote_IP = SSHOperator(
2
task_id='Read_remote_IP',
3
ssh_hook=hook,
4
command="echo remote_IP ",
5
)
6
7
Read_SSH_Output = BashOperator(
8
task_id='Read_SSH_Output',
9
bash_command="echo {{ti.xcom_pull(task_ids='Read_remote_IP')}} ",
10
)
11
How can the SSHOperator Read_remote_IP
return value non-encoded? Also, how can the BashOperator Read_SSH_Output
decode the encoded value?
Advertisement
Answer
My current solution is to introduce another Python operator to convert the string back, please feel free to provide other solutions.
JavaScript
1
14
14
1
import base64
2
from airflow.operators.python import PythonOperator
3
4
def _decode_message(task_name, ti):
5
message = ti.xcom_pull(task_ids=task_name)
6
print(message)
7
return base64.b64decode(message).decode()
8
9
Decode_SSH_Output = PythonOperator(
10
task_id='Decode_SSH_Output',
11
python_callable=_decode_message,
12
op_args=['Read_remote_IP'],
13
)
14