@JTrott The background to the story in Num. 22-24 shows Balaam explaining to the king of Moab, Balak, that God had called and blessed the Israelites. Balaam made that pretty clear. It seems they had plenty of fair warning that God was with Israel. It's too bad they tried swaying Israel toward their god (Baal) and immoral practices rather than trying to learn more about the God of Israel.
@JTrott It was a bad move on the Midianites' part to sway the people of Israel away from worshiping the true God to a false god. It warranted severe punishment. And, by the way, the Israelites who were involved were put to death as well (Num 25). It's harsh by our standards--and rare in Scripture--but it was a crime against the Ultimate Being.