In the end, Napoléon realized it was not small number of fanatics at issue:
"I committed the fault of entering Spain, because I did not know the spirit of the nation. The nobility called me, and the rabble drove me back. This country was not worthy of a prince of my dynasty."