I agree it doesn't help him, but some people are just like that. I know two different people that are exactly like that. There are some folks that don't show emotion or don't show emotion publically. Frankly at this point, I'm not sure how else he could act that wouldn't look contrived or faked. You can't change who you are, no matter the circumstances. My friend who carries the same demeanor, keeps his emotions bottled in except in private. But I would never think he doesn't care about things, because I know him.
That is ultimately the problem here. Nobody here knows Jeremy Irwin. Nobody knows the type of person he is, what his hobbies are, had a deep conversation with him, how he acts around friends, etc. All we see is how he acts in front of a camera and because he's quiet, introverted, etc, he's judged by some as not caring about his daughter. IMO, I don't think that's fair, but then again the public isn't fair.