Great site for DIY... Removing and replacing old grout. First Remove the old grout…. If the grout is already coming out and is fairly loose – Use a grout removal tool found at all home improvement stores. Scrape along the damaged grout line and dig the old grout out of the grout joints.