There are many different canons that feature super strength and varying strength levels between them. So for the sake of fairness, everyone will be averaged out to roughly the same. It would be incredibly hard to stagger the levels, especially with Buffy and vampires as there are a great many canons that feature vampires that would either be significantly stronger or weaker than her.

They won't quite be body-builder strong since even that is hard to place (and some of those men and women can pull trucks with their teeth), but they will be above the human average. But not so much so that they will able to rely entirely on their strength for survival, but more as something that gives them an edge in battle. The best way to think of it is a stat boost, rather than an ability all on it's own.