Flutter Flame collision detection with obstacles

I'm trying to build a game were I have a player and a tiled map with walls and obstacles, the player can move up, right, down, left and I want the player to not go trough the walls or obstacles.

I used the collision system of flutter flame to implement this but I have a bug that its killing me, let me explain the bug that i have with an example

Let's say that the player goes up and collides with a wall, then the player cannot move up anymore
but if he keep colliding with the top edge and start moving right or left he will go through the walls on the left or right when he reaches the edge

class Player extends SpriteComponent
with CollisionCallbacks, HasGameRef<MobArena> {
final MovementJoystick joystick;
bool collided = false;
double speed = 50;
JoystickDirection collidedDirection = JoystickDirection.idle;
Player({required this.joystick}) : super(size: Vector2.all(16));
@override
Future<void> onLoad() async {
await super.onLoad();
add(RectangleHitbox());
}
@override
void update(double dt) {
super.update(dt);
if (!joystick.relativeDelta.isZero()) {
switch (joystick.direction) {
case JoystickDirection.up:
if (collided && collidedDirection == JoystickDirection.up) {
break;
}
position.y += (joystick.relativeDelta * speed * dt)[1];
break;
case JoystickDirection.right:
if (collided && collidedDirection == JoystickDirection.right) {
break;
}
position.x += (joystick.relativeDelta * speed * dt)[0];
break;
case JoystickDirection.down:
if (collided && collidedDirection == JoystickDirection.down) {
break;
}
position.y += (joystick.relativeDelta * speed * dt)[1];
break;
case JoystickDirection.left:
if (collided && collidedDirection == JoystickDirection.left) {
break;
}
position.x += (joystick.relativeDelta * speed * dt)[0];
break;
}
}
}
@override
void onCollision(Set<Vector2> intersectionPoints, PositionComponent other) {
super.onCollision(intersectionPoints, other);
if (other is Obstacle && !collided) {
collided = true;
collidedDirection = joystick.direction;
}
}
@override
void onCollisionEnd(PositionComponent other) {
super.onCollisionEnd(other);
collidedDirection = JoystickDirection.idle;
collided = false;
}
}

If you want to use this type of checking you will have to have a list of collision directions instead of only one, since it can collide from multiple directions at the same time.

In the onCollsionEnd you'll also have to remove the directions that it no longer collides with from that list.

To make it a bit more efficient, you can use onCollisionStart instead of onCollision for checking when you are starting to collide with a new wall.