How correctly set “In-Reply-To” and “Reference” headers in Gmail API

I’m trying to reply a mail I received using Gmail API. I tried following code it appends the sending message to thread in my mailbox but for the receiver it send as a new message. What is the proper way to declare In-Reply-To and Reference headers?

def create_message(origin=None, destination=to, subject=None, msg_txt=None, thr_id=None):
    """Create a message for an email.
    Args:
      origin: Email address of the sender.
      destination: Email address of the receiver.
      subject: The subject of the email message.
      msg_txt: The text of the email message.
      thr_id: the threadId of the message to attach
    Returns:
      An object containing a base64url encoded email object.
    """
    message = MIMEText(msg_txt)
    message['to'] = destination
    message['from'] = origin
    message['subject'] = subject
    raw_msg={'raw': (base64.urlsafe_b64encode(message.as_bytes()).decode())}
    raw_msg['threadId'] =thr_id
    raw_msg['Reference'] = '<CANyAw3CWm33sKL80GMKp-b=[e-mail]>'
    raw_msg['In-Reply-To'] = '<CANyAw3CWm33sKL80GMKp-b=[e-mail]>'
    raw_msg['Message-ID'] = '<CANyAw3CWm33sKL80GMKp-b=[e-mail]>'
    return raw_msg

My main method is as follow,

def main():
    """Canned reply responder using the Gmail API.
    Creates a Gmail API service object and responds to a query with a standard response
    whilst giving it a label to ensure only 1 response per thread is sent
    """

    # get credentials first and create gmail service object
    store = file.Storage('token.json')
    creds = store.get()
    if not creds or creds.invalid:
        flow = client.flow_from_clientsecrets('gmailApiCredentials.json', SCOPES)
        creds = tools.run_flow(flow, store)
    gmail_service = build('gmail', 'v1', http=creds.authorize(Http()))

    # receive email messages
    q = 'subject:this is a test message'
    messages = list_messages_matching_query(gmail_service, user_id,
                                            query=q,
                                            maxResults=1)
    if not messages:
        print("No messages to show")
    else:
        pprint.pprint('Messages to show: {}'.format(messages))

    # get thread of first document - so you can label the thread itself if need be
    thread_id = messages[0]['threadId']
    thread = get_thread(gmail_service, user_id, thread_id)

    msg_id = messages[0]['id']
    message = get_message(gmail_service, user_id, msg_id)

    subject ='Re:this is a test message'
    msg = create_message(destination=to, origin=to,
                         subject=subject,
                         msg_txt='Hai', thr_id=thread_id)
    send_message(gmail_service,"me", msg)
    print("Message Sent")

Answer

The create_message method should be changed as follow to set ‘Reference’ and ‘In-Reply-To’ headers.

def create_message(origin=None, destination=TO, subject=None, msg_txt=None, thr_id=None, msgID=None):
    """Create a message for an email.
    Args:
      origin: Email address of the sender.
      destination: Email address of the receiver.
      subject: The subject of the email message.
      msg_txt: The text of the email message.
      thr_id: the threadId of the message to attach
    Returns:
      An object containing a base64url encoded email object.
    """
    message = MIMEText(msg_txt)
    message['to'] = destination
    message['from'] = origin
    message['subject'] = subject
    message.add_header('Reference', msgID)
    message.add_header('In-Reply-To', msgID)
    raw_msg = {'raw': (base64.urlsafe_b64encode(message.as_bytes()).decode())}
    raw_msg['threadId'] = thr_id
    return raw_msg

The correct msgID can be obtained from following method,

def get_mime_message(service, user_id, msg_id):
    try:
        message = service.users().messages().get(userId=user_id, id=msg_id,
                                                 format='raw').execute()

        msg_str = base64.urlsafe_b64decode(message['raw']).decode()

        mime_msg = email.message_from_string(msg_str)

        return mime_msg
    except errors.HttpError as error:
        print('An error occurred: %s' % error

The msgID can be obtained as follow,

ms = get_mime_message(gmail_service, USER_ID, msg_id)
    msgID = format(ms['Message-ID'])

Do not confuse msg_id with msgID. msg_id is gmail specific and msgID is global. This msgID should be used in ‘Reference’ and ‘In-Reply-To’ headers.