The 'Bourbon point of view' is not limited to the royal family themselves. There were many people, at all levels of French society, who were not Royalists yet considered them a better option for France than Napoleon. You have to look at the wider picture and consider, for instance, the enduring popularity of the Charte of 1814, which Napoleon was forced to copy on his return.
Susan