Well the Polish invasion was the impetus for the declaration of war. What other countries had he invaded before Poland?Really only when they were "forced" into it after Poland was invaded. Before that they let Hitler invade many countries hoping he would "be reasonable", quite a forlorn hope when you think about it.
Regardless of the timeline, the Brits stood up to him. Unlike the French and other European countries.