Yes, I see your point. But although there is philosophy in social science, and philosophy of social science, there is also philosophy that is not social science.

And philosophy's basis is not observation. This might be debatable. highly depends on what you mean with 'observation'. And in the scientific sense of observation, philosophy does not deal with observation. Instead it deals with considering what a subject is, that observes.

All science is partly philosophical, btw. Also historically this can be seen, how sciences slowly defined their own domain as outside the realm of philosophy that was the main domain for a long time.

I do agree philosophy can often be rightfully applied to the social. Just as to the physical.