Match rule json with data json to find value in python3

PYTHON3

Need to Find articleid by applying rules on data

Given a set of rules in JSON format, which includes article IDs and corresponding properties rules.

Eg: rules.json

{
  "rules": [
    {
      "articleId": "art1",
      "properties_rule": [
        {
          "condition": "EQ",
          "logicalOperator": "AND",
          "propertyId": 487,
          "value": "aaaa"
        },
        {
          "condition": "EQ",
          "logicalOperator": "",
          "propertyId": 487,
          "value": "zzzz"
        }
      ],
      
    },
    {
      "articleId": "art2",
      "properties_rule": [
        {
          "condition": "GTE",
          "logicalOperator": "AND",
          "propertyId": 487,
          "value": "bbbb"
        },
        {
          "condition": "LTE",
          "logicalOperator": "",
          "propertyId": 487,
          "value": "eeee"
        }
      ],
      
    },
    {
      "articleId": "art3",
      "properties_rule": [
        {
          "condition": "GTE",
          "logicalOperator": "",
          "propertyId": 487,
          "value": "ffff"
        }
      ],
      
    }
  ]
}

Qs well as a set of data in JSON format, which includes values for certain properties.
Like Eg: data.json

{
  "data": {
    "1": {
      "properties_values": [
        {
          "value": {
            "property_id": 487,
            "property_value": "aaaa",
            "response_id": 1
          }
        }
      ]
    },
    "2": {
      "properties_values": [
        {
          "value": {
            "property_id": 487,
            "property_value": "bbbb",
            "response_id": 2
          }
        }
      ]
    },
    "3": {
      "properties_values": [
        {
          "value": {
            "property_id": 487,
            "property_value": "eeee",
            "response_id": 3
          }
        }
      ]
    }
  }
}

the task is to apply the rules on the data and determine the article ID that matches the rules.

How can we use the provided rules and data JSON to determine the corresponding "articleId" for each entry in the "data" JSON, based on the conditions specified in the "properties_rule" arrays of the rules JSON? The "property_id" field in the "data" JSON corresponds to the "propertyId" field in the "rules" JSON, and the "property_value" field in the "data" JSON corresponds to the "value" field in the "rules" JSON.

Tried this To Solve this But didn't liked so many for loops

To determine the corresponding "articleId" for each entry in the "data" JSON based on the conditions specified in the "properties_rule" arrays of the rules JSON, we can follow the following steps in Python:

1. Load the rules and data JSON into Python dictionaries using the json module.
2. Loop through each entry in the "data" JSON.
3. For each entry, loop through the "rules" JSON to find a matching article ID based on the conditions specified in the "properties_rule" arrays.
4. For each rule, loop through the "properties_values" array in the data entry to find a matching property ID.
5. If a matching property ID is found, check if the property value meets the condition specified in the rule. If it does not meet the condition, move on to the next rule.
6. If all conditions are met, return the article ID associated with the rule.
7. If no matching article ID is found, return a default value or raise an error.

Here is sample code that implements this logic:

import json

# Load the rules and data JSON files into memory
with open('rules.json', 'r') as f:
    rules_data = json.load(f)
rules = rules_data['rules']

with open('data.json', 'r') as f:
    data = json.load(f)
data = data['data']

# Loop through each rule and check if it matches the data
for rule in rules:
    properties_rule = rule['properties_rule']
    articleId = rule['articleId']
    
    # Check if all the conditions in the properties_rule array are satisfied
    matched = True
    for prop_rule in properties_rule:
        propertyId = prop_rule['propertyId']
        value = prop_rule['value']
        condition = prop_rule['condition']
        
        # Check if any of the survey responses match the condition
        response_matched = False
        for key, value_dict in data.items():
            properties_value = value_dict['properties_value']
            for prop_value in properties_value:
                survey_response = prop_value['survey_response']
                if survey_response['property_id'] == propertyId:
                    property_value = survey_response['property_value']
                    if condition == 'EQ' and property_value == value:
                        response_matched = True
                    elif condition == 'GTE' and property_value >= value:
                        response_matched = True
                    elif condition == 'LTE' and property_value <= value:
                        response_matched = True
        # If none of the survey responses match the condition, set matched to False
        if not response_matched:
            matched = False
            break
    
    # If all the conditions are satisfied, return the articleId of that rule
    if matched:
        print("Article ID: ", articleId)
        break

# If none of the rules match the data, return a default articleId
if not matched:
    print("Default Article ID")

But Looking for effecient way of doing it. Don't want so many for loops

> Looking for efficient way of doing it. Don't want so many for loops

You could convert each properties_rule list into a function or lambda expression (accompanied by a list of propertyIds to extract the arguments and an articleId to return).

def rule_to_lambda(rule_dict):
    opRef = {'EQ': '==', 'GTE': '>=', 'LTE': '<=', 'GT': '>', 'LT': '<'}

    lStr1, lStr2, prop_ids, andOr = 'lambda ', ':', [], ''
    for ri, pr in enumerate(rule_dict['properties_rule'], 1):
        prop_ids.append(pr['propertyId'])
        prVal, lStr1 = pr['value'], f'{lStr1}{"" if ri == 1 else ", "}p{ri}'

        if ri != 1: lStr2 += f' {andOr}'
        andOr = 'or' if pr.get('logicalOperator')=='OR' else 'and'

        cond = f'{repr(prVal)} {opRef.get(pr["condition"])} p{ri}'
        lStr2 += f' (p{ri} is not None and {cond})'

    # no rules to check --> return True by default     
    if not prop_ids: lStr1, lStr2 = 'lambda *x', ': True'
 
    return rule_dict['articleId'], prop_ids, eval(lStr1+lStr2)

^{Note that using eval is often considered unsafe, but I think it's alright in this case since you know exactly what can go into lStr1+lStr2 (except for pr['value'], but that's coming from parsed JSON, so it won't be an expression).}

If used on the sample of rules.json in your question,

# import json
# with open('rules.json', 'r') as f: rules = json.load(f)

rules_list = [rule_to_lambda(r) for r in rules['rules']]

then rules_list would [effectively] be the same as

>py
> [
> ('art1', [487, 487], lambda p1, p2: (p1 is not None and 'aaaa' == p1) and (p2 is not None and 'zzzz' == p2)),
> ('art2', [487, 487], lambda p1, p2: (p1 is not None and 'bbbb' >= p1) and (p2 is not None and 'eeee' <= p2)),
> ('art3', [487], lambda p1: (p1 is not None and 'ffff' >= p1))
> ]
>

[You'll still have to loop through every rule for every properties_values in data, but this way you only have to loop through every properties_rule dictionary once, and then you can use the lambda expressions created.]

Since you need the property_value (that corresponds to the property_ids) as arguments for the lambda expressions, you might also want to use a function to find the property_value of a data value based on a property_id:

def get_property_value(prop_id, prop_vals:list):
    for p in prop_vals:
        if prop_id == p['value']['property_id']: 
            return p['value']['property_value']

# {k:get_property_value(487,v['properties_values']) for k,v in data['data'].items()}
# returns {'1': 'aaaa', '2': 'bbbb', '3': 'eeee'}

Now you can loop though data like

# import json
# with open('rules.json', 'r') as f: rules = json.load(f)
# with open('data.json', 'r') as f: data = json.load(f)

# rules_list = [rule_to_lambda(r) for r in rules['rules']]
for k, v in data['data'].items():
    for a, prop_ids, ruleFunc in rules_list:
        pvals = v['properties_values'] 
        pvals = [get_property_value(pi, pvals) for pi in prop_ids]
        if ruleFunc(*pvals):
            print(k, ':', pvals, '---> Article ID:', a)
            break ## print only for the first rule that matches
    else: print(k, '---> Default Article ID')

With your sample data.json and rules.json, this prints Article ID: art3 on all 3 lines, but that's unsurprising since the art1 and art2 rule are impossible unless p1 and p2 are different [and they aren't since prop_ids for both rules is [487,487]].

> py
> 1 : ['aaaa'] ---> Article ID: art3
> 2 : ['bbbb'] ---> Article ID: art3
> 3 : ['eeee'] ---> Article ID: art3
>