Inventory Changed Trigger for a Specific Slot

So I want to trigger a function when the player adds or removes or switches a helmet (with a custom tag) on his head. I tried using the inventory changed advancement criteria for this but I tested 2 things :
"criteria": {
  "head": {
    "trigger": "minecraft:inventory_changed",
    "conditions": {
      "item": {
        "nbt": "{customHead:1b}"
      },
      "player": [
        {
          "condition": "minecraft:entity_properties",
          "entity": "this",
          "predicate": {
            "equipment": {
              "head": {
                "nbt": "{customHead:1b}"
              }
            }
          }
        }
      ]
    }
  }
}

This one works when I add or switch the helmet (but not for removing).
But it also triggers when the head has the helmet and I move an other one on every other slot...
"criteria": {
  "head": {
    "trigger": "minecraft:inventory_changed",
    "conditions": {
      "item": {
        "nbt": "{customHead:1b}"
      }
    }
  }
}

And this one works but it also triggers the function when I move it in the inventory (on every slots)...
So I'm looking for a solution (if possible) that doesn't require me to check at every tick if the player changed his helmet slot (bc I want to do this for every armor slots).
Any help would be much appreciated thanks