Jenkins build failing without updating Xray with the failed status

Please forgive me if this is not the place to ask this question. I’m running python scripts in a Jenkins pipeline from a Jenkinsfile. I am also updating Jira Xray tickets within the Jenkisfile. Behave is being used to validate the test status. If the check fails then the Jenkins build fails without getting the Xray ticket updated with the failure. I’ve attempted to use “try” to capture the failure but have not succeeded in getting the failure to propagate to the Xray ticket.

Would anyone here know where I might find an answer? I would be in your dept.

Jenkinsfile

node() {
    
    def repoURL = '<GitLab URL>/prod-003.git'
    def STC_INSTALL = "/opt/STC_CLIENT/Spirent_TestCenter_5.22/Spirent_TestCenter_Application_Linux/"
    
    try {
        stage("Prepare Workspace") {
            echo "*** Prepare Workspace ***"
            cleanWs()
            env.WORKSPACE_LOCAL = sh(returnStdout: true, script: 'pwd').trim()
            env.BUILD_TIME = "${BUILD_TIMESTAMP}"
            echo "Workspace set to:" + env.WORKSPACE_LOCAL
            echo "Build time:" + env.BUILD_TIME
            sh """
            cd ${env.WORKSPACE_LOCAL}
            rm -fr *
            """
        }
        stage('Checkout Code') {
            echo "*** Checking Code Out ***"
            git branch: 'master', credentialsId: '', url: repoURL
        }
        stage('Executing Tests') {
            if (env.WanModeCheck == "Yes") {
                echo "Executing WAN Mode Change Before FW Upgrade"
                sh """
                /var/lib/jenkins/.pyenv/shims/python WanMode.py -i $modemIP -m $WanMode
                """
                echo "Starting Firmware Upgrade"
                sh """
                cd ${env.WORKSPACE_LOCAL}
                ./ModemUpgrade.sh -i $modemIP -f $FW -p2
                /var/lib/jenkins/.pyenv/shims/behave -f cucumber -o storetarget-bdd/reporting/cucumber.json --junit --format=json -o target/behave.json --junit ./features/PROD-003.feature
                """
            } else {
                echo "#######################n# Skipping WAN Mode Change #n#######################"
            }
            if (env.WanModeCheck == "No") {
                echo "Starting Firmware Upgrade"
                sh """
                cd ${env.WORKSPACE_LOCAL}
                ./ModemUpgrade.sh -i $modemIP -f $FW -p2
                /var/lib/jenkins/.pyenv/shims/behave -f cucumber -o storetarget-bdd/reporting/cucumber.json --junit --format=json -o target/behave.json --junit ./features/fwupgrade.feature
                """
            }
            // Setting variables to use for the Xray Test Execution
            res = sh(returnStdout: true, script: 'awk "/#@/{f=1;next} /#####/{f=0} f" PROD-003-Out.txt | sed -e "s/#//g" -e "s/^ * //g" | tr "n" "%" | sed -e "s/^%%%%%%//g" -e "s/%%$//g" -e "s/%/\\\\Z/g" -e "s/Z/n/g"')
            env.STResults = res.strip()
            model = sh(returnStdout: true, script: 'grep Model: PROD-003-Out.txt')
            env.Model = model.strip()
            wanmode = sh(returnStdout: true, script: 'grep CPE PROD-003-Out.txt')
            env.WanMode = wanmode.strip()
            serialnum = sh(returnStdout: true, script: 'grep Number: PROD-003-Out.txt')
            env.SerialNum = serialnum.strip()
            echo "End of test phase"
        }
        stage('Expose report') {
            echo "*** Expose Reports ***"
            echo "*** Archive Artifacts ***"
            archiveArtifacts "**/cucumber.json"
            echo "*** cucumber cucumber.json ***"
            cucumber '**/cucumber.json'
            junit skipPublishingChecks: true, allowEmptyResults: true, keepLongStdio: true, testResults: 'reports/*.xml'
            cucumber buildStatus: "UNSTABLE",
            fileIncludePattern: "**/cucumber.json",
            jsonReportDirectory: 'reports'
        }
    
        stage('Import results to Xray') {
            echo "*** Import Results to XRAY ***"
            def description = "Jenkins Project: ${env.JOB_NAME}\n\nCucumber Test Report: [${env.JOB_NAME}-Link|${env.BUILD_URL}/cucumber-html-reports/overview-features.html]\n\nJenkins Console Output: [${env.JOB_NAME}-Console-Link|${env.BUILD_URL}/console]\n\nCPE IP: ${modemIP}\n\nCPE FW File Name: ${FW}\n\n${env.STResults}"
            def labels = '["regression","automated_regression"]'
            def environment = "DEV"
            def testExecutionFieldId = 10552
            def testEnvironmentFieldName = "customfield_10372"
            def projectKey = "AARC"
            def projectId = 10608
            def xrayConnectorId = "e66d84d8-f978-4af6-9757-93d5804fde1d"
            // def xrayConnectorId = "${xrayConnectorId}"
            def info = '''{
                "fields": {
                    "project": {
                        "id": "''' + projectId + '''"
                    },
                    "labels":''' + labels + ''',
                    "description":"''' + description + '''",
                    "summary": "''' + env.JOB_NAME + ' ' + env.Model + ' ' + env.WanMode + ' ' + env.SerialNum + ''' Test Executed ''' + env.BUILD_TIME + ''' " ,
                    "issuetype": {
                    "id": "''' + testExecutionFieldId + '''"
                    }
                }
            }'''

            echo info

            step([$class: 'XrayImportBuilder', 
            endpointName: '/cucumber/multipart', 
            importFilePath: 'storetarget-bdd/reporting/cucumber.json', 
            importInfo: info, 
            inputInfoSwitcher: 'fileContent', 
            serverInstance: xrayConnectorId])
        }
    }                                    
    catch(e) {                           
        // If there was an exception thrown, the build failed
        currentBuild.result = "FAILED"
        throw e
    } finally {
        // Success or failure, always send notifications
        echo "Sending final test status to Slack"
        // notifyBuild(currentBuild.result)
    }

}
def notifyBuild(String buildStatus = 'STARTED') {
    // build status of null means successful
    buildStatus =  buildStatus ?: 'SUCCESSFUL'

    // Default values
    def colorName = 'RED'
    def colorCode = '#FF0000'
    def subject = "${buildStatus}: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'"
    def summary = "${subject} (${env.BUILD_URL})"
    def details = """<p>STARTED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]':</p>
      <p>Check console output at &QUOT;<a href='${env.BUILD_URL}'>${env.JOB_NAME} [${env.BUILD_NUMBER}]</a>&QUOT;</p>"""

      // Override default values based on build status
      if (buildStatus == 'STARTED') {
        color = 'BLUE'
        colorCode = '#0000FF'
        msg = "Build: ${env.JOB_NAME} has started: ${BUILD_TIMESTAMP}"
      } else if (buildStatus == 'UNSTABLE') {
        color = 'YELLOW'
        colorCode = '#FFFF00'
        msg = "Build: ${env.JOB_NAME} was listed as unstable. Look at ${env.BUILD_URL} and Report: ${env.BUILD_URL}/cucumber-html-reports/overview-features.html"
      } else if (buildStatus == 'SUCCESSFUL') {
        color = 'GREEN'
        colorCode = '#00FF00'
        msg = "Build: ${env.JOB_NAME} Completed Successfully ${env.BUILD_URL} Report: ${env.BUILD_URL}/cucumber-html-reports/overview-features.html"
      } else {
        color = 'RED'
        colorCode = '#FF0000'
        msg = "Build: ${env.JOB_NAME} had an issue ${env.BUILD_URL}/console"
      }

    // Send notifications
    slackSend (color: colorCode, message: summary)
    slackSend baseUrl: 'https://hooks.slack.com/services/', 
    channel: '#wopr-private', 
    color: colorCode, 
    message: msg,
    teamDomain: '<Slack URL>', 
    tokenCredentialId: 'Jenkins-Slack-Token', 
    username: 'JenkinsAutomation'
}

feature file

Feature: SNMP Firmware Upgrade Test
  @demo @AARC-3428
  Scenario: SNMP Firmware Upgrade Executed against the DUT
    Given ModemUpgrade.sh Script Exists
    When SNMP Firmware Upgrade Executed
    Then I expect Result Pass

step file

from behave import *
import pathlib
from pathlib import Path

@given('ModemUpgrade.sh Script Exists')
def step_impl(context):
    STCFile = pathlib.Path('ModemUpgrade.sh')
    if STCFile.exists():
        print("SNMP Firmware Upgrade file exists")
        pass
    # else:
    #     print("SNMP Firmware Upgrade file does not exists")
    #     assert context.failed

@when('SNMP Firmware Upgrade Executed')
def step_impl(context):
    path_to_file = 'PROD-003-Out.txt'
    path = Path(path_to_file)
    if path.is_file():
        print(f'Output file {path_to_file} exists')
    else:
        print(f'Output file {path_to_file} does not exists')

@then('I expect Result Pass')
def step_impl(context):
    Result = False
    with open("PROD-003-Out.txt") as FwUpgradeResults:
        for line in FwUpgradeResults:
            if 'Upgrade Status: Passed'.lower() in line.strip().lower():
                Result = True
                break
            else:
                Result = False
                break
    if Result is False:
        print("Error: Upgrade Failed")
        assert context.failed

The suggestion of using || /usr/bin/true appears to have worked for the above mentioned code. Now I have a second instance where my Python test is throwing an exception when the DUT fails DHCP bind

def wait_for_dhcp_bind():
    try: 
        stc.perform("Dhcpv4BindWait", objectlist=project)
    except Exception:
        raise Exception("DHCP Bind Failed")

I attempted to add the same after the Python script but the Jenkins build fails without the Xray test getting updated with a failure.

Here is what this looks like in the Jenkinsfile

            echo "Starting Speed Test"
            // def ModemMac = sh(returnStdout: true, script: './ModemUpgrade.sh -i ${modemIP} -f mac')
            sh """
                export STC_PRIVATE_INSTALL_DIR=${STC_INSTALL}
                cd ${env.WORKSPACE_LOCAL}
                /var/lib/jenkins/.pyenv/shims/python SpeedTest.py -d $dsp -u $usp -i $iterations -x $imix -f $frames -m $ModemMac || /usr/bin/true
                /var/lib/jenkins/.pyenv/shims/behave -f cucumber -o storetarget-bdd/reporting/cucumber.json --junit --format=json -o target/behave.json --junit ./features/speedtest.feature || /usr/bin/true
            """

Answer

Your case should be easy to fix. Behave utility returns exit code 1 if tests fails..

Just add this to the end of your behave command || /usr/bin/true (please make sure of the path of the “true” command). This will make your command to always return true even if some problems exist with behave.

So your overall command should be something like:

/var/lib/jenkins/.pyenv/shims/behave -f cucumber -o storetarget-bdd/reporting/cucumber.json --junit --format=json -o target/behave.json --junit ./features/PROD-003.feature || /usr/bin/true