Rights Reversion Regrets

I've made hundreds of thousands of dollars in advances and royalties in Trad publishing. Half my list consists of traditionally agented titles. However, when it comes time to ask for your rights back for any particular book or books, the publishers can be really difficult to work with, even if they no longer have any interest in promoting your titles.