Django filter on many to many with having count

After a lot of try, I am not able to translate this kind of sql queries into django filtering.
Basically, it is retrieving all users with at least x selected tags (and other filters).
In the case of two tags, users need to have both tags, I don't want to retrieve users with at least one of the tag.

Here are my definitions:

class User(models.Model):
    name = models.CharField(max_length=64)
    tags = models.ManyToManyField(Tag)
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)
    
class Tag(models.Model):
    name = models.CharField(max_length=128)
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

SQL query will look like:

SELECT user.id
FROM user
INNER JOIN "user_tags" ON ("user"."id" = "user_tags"."user_id")
AND "user_tags"."tag_id" in (8, 163) group by "user"."id", user_tags.user_id having count(*) = 2
ORDER BY user.id DESC LIMIT 100;

As you can see, the having count(*) part is done on the created m2m table user_tags, and the filtering of tags also "user_tags"."tag_id" in (8, 163)

Is it achievable in a django way?
All my attempts use user and tab tables, never the created table.

Thanks,

EDIT: passing @preeti-y as right answer as it looks like this is the way to go.
But for my specific case and for performances reasons, I directly used a RawSQL method allowing me to correctly pass it with other filters.

Q(id__in=RawSQL(
                    '''SELECT U0.id FROM "user" U0 
                    INNER JOIN "user_tags" U1 ON (U0."id" = U1."user_id") 
                    WHERE U1."tag_id" IN %s
                    GROUP BY U0."id" 
                    HAVING COUNT(U1."user_id") = %s''', (tuple(value), len(value),)))

Based on your query, in Django way, you can do Count() with filter argument in the annotation like this:

User.objects.annotate(
  matched_tag_count=Count('tags', filter=Q(tags__in=[8, 163]))
).filter(matched_tag_count=2)