Skip to content
Advertisement

How to compare 2 different csv files and output the differences without CSV-diff

I have 2 CSVs which are New.csv and Old.csv shown below:

New.csv

longName,shortName,eventType,number,severity,msg,note,om,action,probableCause,correctiveAction,alarmName,clearCondition,logThrottleCount,upgradeCritical,complexDescTemplate
ACTAGENT201,ACAT201,RES,1,INFO,Accounting is enabled upon this NE.,This log is generated when setting a Session Manager's AM from <none> to a valid AM.,"On all instances of this Session Manager, the <NE_Inst>:<AM>:STD:acct OM row in the  StdRecordStream group will appear and start counting the recording units sent to the configured AM.
                   On the configured AM, the <NE_inst>:acct OM rows in RECSTRMCOLL group will appear and start counting the recording units received from this Session Manager's instances.",,,,,,,,
ACTAGENT202,ACAT202,RES,2,ALERT,Accounting is disabled upon this NE.,This log is generated when setting a Session Manager's AM from a valid AM to <none>.,"On all instances of this Session Manager, the <NE_Inst>:<AM>:STD:acct OM row in the StdRecordStream group that matched the previous datafilled AM will disappear.
                   On the previously configured AM, the  <NE_inst>:acct OM rows in RECSTRMCOLL group will disappear.","If you do not intend for the Session Manager to produce accounting records, then no action is required.  If you do intend for the Session Manager to produce accounting records, then you should set the Session Manager's AM to a valid AM.",,,,,,,
ACODE801,AC801,ADMIN,1,MINOR,"Configured data for audiocode server updated: $1,",,,,INFORMATION_MODIFICATION_DETECTED,None. Acknowledge/Clear alarm and deploy the audiocode server if appropriate.,Audiocode Server Updated,,,,
ACODE802,AC802,ADMIN,2,MINOR,"Deployment for audiocode server failed: $1. Reason: $2.,
                         ,",,,,CONFIG_OR_CUSTOMIZATION_ERROR,Check the configuration of audiocode server. Acknowledge/Clear alarm and deploy the audiocode server if appropriate.,Audiocode Server Deploy Failed,,,,
ACODE102,AC102,COMM,2,CRITICAL,"Far end LOF (a.k.a., Yellow Alarm). Trunk (DS1 Number): $1.,",This alarm indicates the Trunk Framing settings on the connected PSTN switch do not match those provisioned on the Audiocodes Mediant 2k.,,,LOSS_OF_FRAME,Check that the far end is configured for the proper framing.,Far end LOF,Far end is correctly configured for proper framing.,,,
ACODE103,AC103,COMM,3,CRITICAL,"Near end sending LOF Indication. Trunk (DS1 Number): $1.,",,,,LOSS_OF_FRAME,Check that the Audiocodes gateway is configured for the proper framing.,Near end sending LOF Indication,Gateway is correctly configured for proper framing.,,,
ACODE104,AC104,COMM,4,CRITICAL,"Far end sending AIS. Trunk (DS1 Number): $1.,",,,,UNDERLYING_RESOURCE_UNAVAILABLE,Check that the far end is configured properly.,Far end sending AIS,Misconfiguration in far end is corrected.,,,
ACODE105,AC105,COMM,5,CRITICAL,"Near end sending AIS. Trunk (DS1 Number): $1.,","The port on the gateway is unable to tell the trunk is in-service.  At this point, the trunk will attempt to send an Alarm Indication Signal (AIS).",,,UNDERLYING_RESOURCE_UNAVAILABLE,Check that the gateway is configured properly.,Near end sending AIS,Misconfiguration in the gateway is corrected.,,,
ACODE106,AC106,COMM,6,CRITICAL,"Near end LOF (a.k.a., Red Alarm). Trunk (DS1 Number): $1.,",,,,LOSS_OF_FRAME,Check the gateway to make sure that the configuration is correct.,Near end LOF,Misconfiguration in gateway is corrected.,,,
ACODECodeNew,AC107,COMM,6,CRITICAL,"Near end LOF (a.k.a., Red Alarm). Trunk (DS1 Number): $1.,",,,,LOSS_OF_FRAME,Check the gateway to make sure that the configuration is correct.,Near end LOF,Misconfiguration in gateway is corrected.,,,
ACODECodeNew2,AC108,COMM,6,CRITICAL,"Near end LOF (a.k.a., Red Alarm). Trunk (DS1 Number): $1.,",,,,LOSS_OF_FRAME,Check the gateway to make sure that the configuration is correct.,Near end LOF,Misconfiguration in gateway is corrected.,,,

Old.csv

longName,shortName,eventType,number,severity,msg,note,om,action,probableCause,correctiveAction,alarmName,clearCondition,logThrottleCount,upgradeCritical,complexDescTemplate
ACTAGENT201,ACAT201,RES,1,INFO,Accounting is enabled upon this NE.,This log is generated when setting a Session Manager's AM from <none> to a valid AM.,"On all instances of this Session Manager, the <NE_Inst>:<AM>:STD:acct OM row in the  StdRecordStream group will appear and start counting the recording units sent to the configured AM.
                   On the configured AM, the <NE_inst>:acct OM rows in RECSTRMCOLL group will appear and start counting the recording units received from this Session Manager's instances.",,,,,,,,
ACTAGENT202,ACAT202,RES,2,ALERT,Accounting is disabled upon this NE.,This log is generated when setting a Session Manager's AM from a valid AM to <none>.,"On all instances of this Session Manager, the <NE_Inst>:<AM>:STD:acct OM row in the StdRecordStream group that matched the previous datafilled AM will disappear.
                   On the previously configured AM, the  <NE_inst>:acct OM rows in RECSTRMCOLL group will disappear.","If you do not intend for the Session Manager to produce accounting records, then no action is required.  If you do intend for the Session Manager to produce accounting records, then you should set the Session Manager's AM to a valid AM.",,,,,,,
ACODE801,AC801,ADMIN,1,MINOR,"Configured data for audiocode server updated: $1,",,,,INFORMATION_MODIFICATION_DETECTED,None. Acknowledge/Clear alarm and deploy the audiocode server if appropriate.,Audiocode Server Updated,,,,
ACODE802,AC802,ADMIN,2,MINOR,"Deployment for audiocode server failed: $1. Reason: $2.,
                         ,",,,,CONFIG_OR_CUSTOMIZATION_ERROR,Check the configuration of audiocode server. Acknowledge/Clear alarm and deploy the audiocode server if appropriate.,Audiocode Server Deploy Failed,,,,
ACODE102,AC102,COMM,2,CRITICAL,"Far end LOF (a.k.a., Yellow Alarm). Trunk (DS1 Number): $1.,",This alarm indicates the Trunk Framing settings on the connected PSTN switch do not match those provisioned on the Audiocodes Mediant 2k.,,,LOSS_OF_FRAME,Check that the far end is configured for the proper framing.,Far end LOF,Far end is correctly configured for proper framing.,,,
ACODE103,AC103,COMM,3,CRITICAL,"Near end sending LOF Indication. Trunk (DS1 Number): $1.,",,,,LOSS_OF_FRAME,Check that the Audiocodes gateway is configured for the proper framing.,Near end sending LOF Indication,Gateway is correctly configured for proper framing.,,,
ACODE104,AC104,COMM,4,CRITICAL,"Far end sending AIS. Trunk (DS1 Number): $1.,",,,,UNDERLYING_RESOURCE_UNAVAILABLE,Check that the far end is configured properly.,Far end sending AIS,Misconfiguration in far end is corrected.,,,
ACODE105,AC105,COMM,5,CRITICAL,"Near end sending AIS. Trunk (DS1 Number): $1.,","The port on the gateway is unable to tell the trunk is in-service.  At this point, the trunk will attempt to send an Alarm Indication Signal (AIS).",,,UNDERLYING_RESOURCE_UNAVAILABLE,Check that the gateway is configured properly.,Near end sending AIS,Misconfiguration in the gateway is corrected.,,,
ACODE106,AC106,COMM,6,CRITICAL,"Near end LOF (a.k.a., Red Alarm). Trunk (DS1 Number): $1.,",,,,LOSS_OF_FRAME,Check the gateway to make sure that the configuration is correct.,Near end LOF,Misconfiguration in gateway is corrected.,,,

If there is a longName (first column) in in the new.csv that is not in the old.csv, I would like that entire new.csv row to be appended to the changes.csv. The current code below is done with csv-diff which seems be a bit glitchy and hard to work with. Is there an alternative way it could be done without csv-diff?

Output should basically look like this with the above CSV’s: enter image description here

CSV-diff code:

from csv import DictWriter
def deltaFileMaker():
    from csv_diff import load_csv, compare
    diff = compare(
        load_csv(open("Old.csv",encoding="utf8"), key="longName"),
        load_csv(open("New.csv",encoding="utf8"), key="longName")
    )
    print(diff)

    # Get all the row headers across all the changes
    headers = set({'change type'})
    for key, vals in diff.items():
        for val in vals:  # Multiple of the same difference 'type'
            headers = headers.union(set(val.keys()))

    # Write changes to file
    with open('changes.csv', 'w', encoding='utf-8') as fh:
        w = DictWriter(fh, headers)
        w.writeheader()
        for key, changes in diff.items():
            for val in changes:  # Add each instance of this type of change
                val.update({'change type': key})  # Add 'change type' data
                w.writerow(val)



deltaFileMaker()

Advertisement

Answer

You can build a set of the longNames in the old csv and then check the new values against it.

import csv

def update_csv(old_csv, new_csv, changes_csv):
    with open(old_csv, newline="") as old_fp:
        reader = csv.reader(old_fp)
        header = next(csv.reader)
        old_long_names = {row[0] for row in reader}
    with open(new_csv, newline="") as new_fp:
        new_reader = csv.reader(new_fp)
        with open(changes_csv, "w", newline="") as changes_fp:
            writer = csv.writer(changes_fp)
            header.insert(0, "Change Type")
            writer.write_row(header)
            new_type = ["New"]
            csv.writer(changes_fp).writerows((new_type + row)
                for row in csv.reader(new_fp)
                if row[0] not in old_long_names)

update_csv("old.csv", "new.csv", "changes.csv")
User contributions licensed under: CC BY-SA
8 People found this is helpful
Advertisement