AirPods increasing metallic noises

I have an issue with my AirPods Pro, both AirPods Pro 3rd generation and AirPods 4th generation.

When I use any kind of noise cancellation or adaptive modes, when I am, like, in the gym and there is a metal touching another metal, or, like, for example, when I’m locking the door and you hear this loud click, but it seems that AirPods amplify these sounds like so it’s twice or even three times as loud as it should be. So it’s kind of even painful and the only way to fix it is to disable all the noise features.