Skip to content
Advertisement

Decode UTF-8 encoded Xcom value from SSHOperator

I have two Airflow tasks that I want to communicate. The SSHOperator returns the last line printed, in this case, “remote_IP”. However, the SSHOperator’s return value is encoded using UTF-8.

Read_remote_IP = SSHOperator(
    task_id='Read_remote_IP',
    ssh_hook=hook,
    command="echo remote_IP ",
)

Read_SSH_Output = BashOperator(
    task_id='Read_SSH_Output',
    bash_command="echo {{ti.xcom_pull(task_ids='Read_remote_IP')}} ",
)

How can the SSHOperator Read_remote_IP return value non-encoded? Also, how can the BashOperator Read_SSH_Output decode the encoded value?

Advertisement

Answer

My current solution is to introduce another Python operator to convert the string back, please feel free to provide other solutions.

import base64
from airflow.operators.python import PythonOperator

def _decode_message(task_name, ti):
    message = ti.xcom_pull(task_ids=task_name)
    print(message)
    return base64.b64decode(message).decode()

Decode_SSH_Output = PythonOperator(
    task_id='Decode_SSH_Output',
    python_callable=_decode_message,
    op_args=['Read_remote_IP'],
)
User contributions licensed under: CC BY-SA
3 People found this is helpful
Advertisement