Need some more help with right click detecting lightsaber pack

Hello,
I made a post earlier but still had some questions. I wanted to make a lightsaber datapack to work with my resource pack. The function that it turns on/off when right clicking the fungus on a stick does work but I have a small issue and I hope that there is a way to fix this small issue.
The issue is that when I have both sabers turned off and then turn on the blue one and select the green one after that one that that one will also turn on without right clicking. Same thing happens when they are both on and turn off one and select the other one. https://gyazo.com/556faffc85e0603b6309b4e4a631185d