There's a quick flash of pain, when Elda points out that Misaki is the one on the outside, but he knows it's true. He doesn't consider her a part of the rest of this.

"She..left me," Togusa hesitantly admits. "Not her fault. We just woke up one day and realized so much had changed." He doesn't even know who or what to blame any more, so he shoves it all down for Hikari's sake. To still talk to Misaki, cheer on her career from afar, and do what he can to raise their daughter. "There's probably some cruelty on my part, through it all. But I think what I was doing before was worse."

For her part, Misaki doesn't contradict anything Togusa is saying.