Open Policy Agent - partial policy evaluation and http.send

With OPA, I would like to implement data query protection and filtering. There is this blog article about this:
https://blog.openpolicyagent.org/write-policy-in-opa-enforce-policy-in-sql-d9d24db93bf4

We are working with a module / microservice architecture where it is necessary to request data from different modules / services for policy evaluation.

My goal is to evaluate the policies defined in Rego partially and to request the required data from other services via http.send (https://www.openpolicyagent.org/docs/latest/policy-reference/#builtin-http-httpsend). The result of the partially evaluated policies should then be used to generate SQL queries that ensure that users only get the data they have access to according to the policies. The result of the partially evaluated policies should contain all necessary values for the filtering of the data model of the service, so that no further queries to other services are necessary.

Policy Example:

allow {
  input.method = "GET"
  input.path = ["entity"]
  allowed[entity]
}

allowed[entity] {
   entities = data.entities[_] # data.entities is unknown
   userEntities2 := http.send({
      "method": "GET",
      "url": concat("", ["/service2/users/", input.userId, "/entity2"]),
   }).body
   userEntities2Ids := [ui.id | ui = userEntities2[_]]
   entity.entities2Ids[_] = userEntities2Ids[_]
}

Workflow / Architecture sketch:

Unfortunately, the HTTP requests do not seem to be executed and evaluated when v1/compile is called. If I call /v1/data, then the result is {"allow": false,"allowed": [],"denied": []}. How can I solve the problem?

> Unfortunately, the HTTP requests do not seem to be executed and evaluated when v1/compile is called.

You're correct, http.send is excluded from partial eval (PE). The reason is that it was meant as a way to pull in the most recent data during policy evaluation. Thinking about it like that, if you pulled in data for PE, the generated queries might not represent the state of the world by the time they're fully evaluated.

Of course there are options, still:

1. provide the data by your own means -- overloading input, or by providing something in data.external with the store used for PE
2. run the partial evaluation through golang code -- I believe you should be able to control the list of unsafe builtins (but I'm not 100% sure about this)
3. through golang code, you could write your own builtin. It wouldn't be known (and thus considered unsafe), and as long as all arguments are defined, the call should be executed during partial eval.

I can elaborate, but I'll need to know more about your scenario for that.