azureml score.py not reading additional json file

I have a json file which I am using in score.py however, it is not being found. When making a post request to the endpoint I get the following error “No such file or directory: ‘/var/azureml-app/model_adjustments.json'”

json file is in the same folder as score.py and calling it from a script

path_to_source="path/example"
inf_conf = InferenceConfig(entry_script="score.py",environment=environment,source_directory=path_to_source)

in my score.py file i have the following

def init():
    global model


# note you can pass in multiple rows for scoring
def run(raw_data):
    
    try:
        # parse the features from the json doc
        dat = json.loads(raw_data)

        #deserialize the model file back into a sklearn mode
        model_name = "{0}_{1}_{2}".format(dat["col1"], dat["col2"], dat["col3"])
        model_path = Model.get_model_path(model_name = model_name)
        model = load(model_path)
        
        # parse the savings json file
        current_directory = os.getcwd()
        file_name="model_adjustments.json"
        json_file=os.path.join(current_directory,file_name)

        with open(json_file, "r") as fr:
            adjustments_dat = json.loads(fr.read())
        modelAdjustment = adjustments_dat[model_name]

        # create a dataframe of the features
        dfPredict = pd.DataFrame(
            columns = [
                "feature1",
                "feature2",
                "feature3",
                "feature4",
                "feature5"
            ]
        )
   
        
        # predict the outcome for these features
        prediction = model.predict(dfPredict)
        Predicted_val = prediction[0]
        #prediction + adjustment for buffer
        final_val= Predicted_val+modelAdjustment 
        
        return {
            "final_val": final_val}
        
    except Exception as e:
        error = str(e)
        return {"debug": -1, "error": error}

I know its to do with the json file because when I remove everything to do with it there is no error when doing a post request. I get back a value like I expected. not sure why its not reading the json file

Answer

I have had not time to try this out, but I would assume that the score.py file (and any files in the same directory) will not be in the os.getcwd().

I would try to use __file__ to get the path of score.py:

so, instead of:

current_directory = os.getcwd()
file_name="model_adjustments.json"
json_file=os.path.join(current_directory,file_name)

rather try:

import pathlib
model_directory = pathlib.Path(os.path.realpath(__file__)).parent.resolve()
file_name="model_adjustments.json"
json_file=os.path.join(model_directory,file_name)

(Note: realpath is there to make sure symlinks are properly resolved)

Advertisement

Answer