That's a very unfortunate phrasing, to say that Gerry felt he was "lucky" to not have to eat with his children. If that is exactly what Gerry said, that is truly unfortunate that he would not recognize how that would sound--ditto for the writer as well.
From what I have read about Gerry's upbringing, I think that in all likelihood, Gerry's father did not have much time or energy to be what we would consider a "good dad" today. Back then, a good father went to work, lived a decent life, provided for his family, but was not a "hands on" kind of father. If he stopped by the pub or bar on the way home a night or two, that was understandable. (In my childhood it was hunting or fishing.) Today, a father is expected to be outstanding in his career, a soccer coach on the weekend, and a great guy in the meantime. I really do believe that Gerry worked hard at his career, stepped in to play or be with the children for short periods, and then considered himself as deserving of some downtime in his own pursuits. He's no different than many other people in that respect, but he is different in that he doesn't seem to understand how it sounds to say that publicly.
He doesn't seem to understand at all that many people really enjoy being with their children and regard themselves as the lucky ones.
And Gerry probably considers himself a very good father. He doesn't run around on his wife, he provides a good living for his family, and he gives them his attention even though it's for limited amounts of time. That's the best he can do and considering his own childhood, it seems pretty good to him.
That he doesn't see how it sounds to say "lucky he didn't have to be with his children" is more indicative of Gerry's inability to understand how differently other people view his words and actions.
It is really sad to me because I don't think he would ever understand that anyone else would call his words "selfish."