I have two Airflow tasks that I want to communicate. The SSHOperator returns the last line printed, in this case, “remote_IP”. However, the SSHOperator’s return value is encoded using UTF-8.
Read_remote_IP = SSHOperator( task_id='Read_remote_IP', ssh_hook=hook, command="echo remote_IP ", ) Read_SSH_Output = BashOperator( task_id='Read_SSH_Output', bash_command="echo {{ti.xcom_pull(task_ids='Read_remote_IP')}} ", )
How can the SSHOperator Read_remote_IP
return value non-encoded? Also, how can the BashOperator Read_SSH_Output
decode the encoded value?
Advertisement
Answer
My current solution is to introduce another Python operator to convert the string back, please feel free to provide other solutions.
import base64 from airflow.operators.python import PythonOperator def _decode_message(task_name, ti): message = ti.xcom_pull(task_ids=task_name) print(message) return base64.b64decode(message).decode() Decode_SSH_Output = PythonOperator( task_id='Decode_SSH_Output', python_callable=_decode_message, op_args=['Read_remote_IP'], )