"I think all of us should speak up if we think something is morally wrong," Rivers said.