There were some in Spain who supported the French, I agree; they were in the tiny minority.
That the Portuguese suffered under the British is a bit of an exaggeration; I assume you are referring to the creation of the bare zone in front of the lines of Torres Vedras? Needs must when the devil drives, and guess who was in the driving seat at that point.