My attribute modifiers arn't stacking

In my datapack i have a custom weapon called a spear and an item called an attack charm. when in your off hand the attack charm uses attribute modifers one of which doubles your attack damage however when i combined the charm with my spear instead of doubling the spears damage instead the game choose the last item equipt as the item to take damage from. E.g i put my charm in my off hand and then hold out my spear in my main hand the game uses my spears damage and ignores the charm. this doesn't happen with regular weapons and tools.