I don't know if there is a definitive answer. It seems to depend on the starship. The NX-01 Enterprise ended up in the Smithsonian as mentioned by in the ENT episode "These are the Voyages....". I would assume most ended up in junkyards or dismantled for scraps at the various Federation shipyards. The Enterprise visits a scrap yard, called Zakdorn Surplus Depot Z-15, in the TNG episode Unification Part I, while looking Vulcan ship that was supposed to have been decommissioned.
To Starship Heaven. Klingon Ships go to Sto'Vo'Kor. :)
If it is anything like real life, they are either sold for scrap, donated to museums, or placed in mothballs