How can I check if the player loses the Haste effect through advancements?

I currently have the following:
  "criteria": {
    "removed_haste_effect": {
      "trigger": "minecraft:effects_changed",
      "conditions": {
        "player": [
          {
            "condition": "minecraft:inverted",
            "term": {
              "condition": "minecraft:entity_properties",
              "entity": "this",
              "predicate": {
                "effects": {
                  "minecraft:haste": {}
                }
              }
            }
          }
        ]
      }
    }
  },
  "requirements": [
    [
      "removed_haste_effect"
    ]
  ],
  "rewards": {
    "function": "minecraft:effect_removed/haste"
  }
}```
The function is a simple command that revokes the advancement and says the player lost the haste effect. The function only runs when the effect runs out of duration, not when /cleared or when the player drinks milk.