Make Allay follow a player without any items

I'm not sure how I could make an allay follow the player even when not having any items. I've tried "/data modify entity @e[type=minecraft:allay,limit=1,sort=nearest,distance=..5] Brain.memories."minecraft:liked_player".value set from entity @s UUID", and I can see it sets in the data, but there isn't any effect on the AI 1.20.1