Manifest files don't "do" anything. So you can remove it, but that won't affect the system.
Manifest files are read so that a "service" can be created. Once created, the service is independent of the manifest.
You can use svccfg to delete a service or service instance (I think it has to be offline).
--
Darren