How to handle line breaks in a templated file included in a Jinja2 JSON template

I am writing a Jinja2 template to output a JSON DAG where one value is a string containing a list of commands from an external shell script, (containing templated variables. This works if the commands in the script are semicolon-separated on a single line:

echo 'hello'; echo 'world';

But, it fails to render the template when formatted with each command on its own line:

echo 'hello';
echo 'world';

JSONDecodeError: Invalid control character at: line 2 column 29 (char 30)` where character 30 is the line break.

I understand this is because JSON does not support multi-line strings. I am using Jinja2’s {% include path %} to load the file but am not sure how to escape the new lines before rendering them. I was not able to pipe the include output through a Python replace successfully: {% include path | replace("n", " ") %}

I also tried putting the include in a macro with the replace calling inside or outside the macro.

Here is a full example of the template and rendering code.

multi_line.sh

echo 'hello';
echo '{{variable}}';

variables.json

{
    "variable": "world!"
}

template.j2

{
   {# Render included file into single line #}
   "multi": "{% include './multi_line.sh' %}"
}

template_renderer.py

from jinja2 import Environment, FileSystemLoader
import json

with open("./variables.json") as variables_file:
    variables_json = json.load(variables_file)

loader = FileSystemLoader("./")
env = Environment(loader=loader)
template = env.get_template("./template.j2")
rendered_template = template.render(variables_json)
json_output = json.loads(rendered_template)

print(json.dumps(json_output, indent=2))

Answer

The solution for me was to wrap the include in a block assignment and then pipe the results into replace to remove occurrences of n.

    {% set multi_line_script %}
    {% include 'multi_line.sh' %}
    {% endset %}
    
    {
        "multi": "{{ multi_line_script | replace("n", " ") }}"
    }

Advertisement

Answer