Focus on what you want. You are living your life and are responsible to the woman you see in the mirror. You don't tell others how to live their lives. Letting them dictate your actions will never make you happy!
"You forgive people simply because you still want them in your life". but are they worth it or are they slowly eating away at your happiness? Life is short changes need to be made or you'll have to limit your time with them.