I believe Ferdinand had more than one brother, and that Napoleon took pains to secure them before the abdication.
I have read that the brothers of Louis XVI refused to recognise his actions while he was in captivity, both because he was under duress and because he could not sign away the prerogatives of the monarchy.
I think Napoleon's claim did rest largely on having the biggest army!
regards,
Susan