Make the piglins and pillagers attack you directly. Who goes around with an unloaded crossbow? Any mob that uses a crossbow will just shoot you.
