Rocket Chip - Access Exception on Page Table Walk

I am trying to integrate a RoCC accelerator with RocketChip / Chipyard. Given a virtual address, the accelerator should translate it to a physical address, read some data from memory and then start a black box accelerator (e.g. AES).
I looked at some examples and (so far) mostly followed the TranslatorExample included in RocketChip. However, the PTW request throws an access exception (ptw.resp.bits.ae_ptw is set). What am I missing?

This is my RoCC code:

class AESAcceleratorModule(outer: MyAESAccelerator)(implicit p: Parameters)
extends LazyRoCCModuleImp(outer) with HasCoreParameters {
val busy = RegInit(Bool(false))
val req_rd = Reg(Bits(width = 5))
val funct = io.cmd.bits.inst.funct
val loadKey = funct === UInt(0)
val loadPlain = funct === UInt(1)
// Memory Stuff
val pte = Reg(new PTE)
val input_addr = Reg(UInt(coreMaxAddrBits.W))
val input_offset = input_addr(pgIdxBits - 1, 0)
val input_vpn = input_addr(coreMaxAddrBits - 1, pgIdxBits)
val input = Reg(UInt(64))
// State Handling
object CtrlState extends ChiselEnum {
val sIdle, sLoadKey, sPTWalk, sPTResp, sResp = Value
}
val cstate = RegInit(CtrlState.sIdle)
// Default Signal Assign:
io.cmd.ready := (cstate === CtrlState.sIdle)
io.busy := (cstate =/= CtrlState.sIdle)
io.resp.valid := (cstate === CtrlState.sResp || cstate === CtrlState.sLoadKey)
io.resp.bits.rd := req_rd
io.resp.bits.data := 1.U
// Translate Address
when(io.cmd.fire && loadPlain){
input_addr := io.cmd.bits.rs1
printf("Input Address: %x\n", io.cmd.bits.rs1)
cstate := CtrlState.sPTWalk
}
private val ptw = io.ptw(0)
ptw.req.valid := (cstate === CtrlState.sPTWalk)
ptw.req.bits.valid := true.B
ptw.req.bits.bits.addr := input_vpn
when (ptw.req.fire()) { cstate := CtrlState.sPTResp }
when (cstate === CtrlState.sPTResp && ptw.resp.valid) {
printf("Got PT Response:\n")
pte := ptw.resp.bits.pte
cstate := CtrlState.sResp
when(ptw.resp.bits.ae_ptw){
printf("Access Exception") // <-- This happens
}
when(ptw.resp.bits.pf){
printf("Page Fault")
}
}
// Data is 0xFFFF...FF - presumably due to the access exception
io.resp.bits.data := Mux(pte.leaf(), Cat(pte.ppn, input_offset), -1.S(xLen.W).asUInt)
when (cstate === CtrlState.sResp && io.resp.fire){
input := Mux(pte.leaf(), Cat(pte.ppn, input_offset), -1.S(xLen.W).asUInt)
cstate := CtrlState.sIdle 
}
// Be busy when have pending memory requests or committed possibility of pending requests
io.interrupt := Bool(false)
// Set this true to trigger an interrupt on the processor (please refer to supervisor documentation)
io.mem.req.valid := Bool(false)
}

And the config file:

class MyAESAcceleratorConfig extends Config(
new WithMyAESAccelerator ++
new RocketConfig)
class WithMyAESAccelerator extends Config ((site, here, up) => {
case BuildRoCC => up(BuildRoCC) ++ Seq(
(p: Parameters) => {
val aes = LazyModule.apply(new MyAESAccelerator(OpcodeSet.custom2)(p))
aes
}
)
})

This is the C code I am executing which does not return the desired result:

#include <stdio.h>
#include "../../tests/rocc.h"
int main(void)
{
uint64_t result = 0;
uint64_t myint =   0xAAAAAAAAAAAAAAAA;
printf("%lx\n", &myint);
asm volatile("fence");
ROCC_INSTRUCTION_DSS(2, result, &myint, 0, 1);
printf("The result of func1 was %lx\n", result);
return 0;
}

In case you haven't figured this out, I assume you are running baremetal C tests. In baremetal VM is not used. This can be found out by checking the ptbr.mode signal (see RISC-V privileged spec section 4).
If you want to test PTW, you can run spike or VCS/verilator with proxy kernel.