He knows he's not being forced out, but even so, the suggestion makes him feel sick to his stomach. The thought of leaving her is bad enough, but the thought of trying to spend time with Ianto, when all he can think of his her is just so much more.

All that's in his mind is the scent of her perfume, the softness of her skin, the way her hair drapes across her neck. No, he couldn't do that, not to himself and not to Ianto

That, and there's an almost unconscious part of him that holds out for more. Holds out for her to give in though his rational mind doesn't expect she will.

"I don't want to leave." He tells her, his voice sad.