We DO know that AJ was in that house alone, after her mom and the younger sibs left for work/school. At about 2:30, the mom and at least one of the sibs returned home together, after work and school was finished. The sib said AJ was not home at that time. She texted AJ: "Where are you?" AJ (probably NOT AJ, but whoever had her phone) replied by text: "With friends."
This is the scenario as I envision it:
WH was supposed to be working alone in Franklin, VA ("out-of-town") that day. This is location where he had been working prior to AJ's coming home for spring break. Sometime during the morning, the Boss probably tried to make phone contact with WH, to check in with him at the jobsite. WH was unreachable -- probably had turned his phone off (IMO). This probably raised Boss' dander.
After his rendezvous with AJ, WH drove back to Franklin. He turned his phone back on. When the Boss was finally able to reach WH, he let him have it. [Boss: "Where the ****!! were you?!"] And WH said he'd taken a "late lunch."
WH tells a partial truth: (WH: "my boss reamed me"). Yes, the Boss reamed him, but not just for being late getting back from lunch. He did not know WH's whereabouts that entire morning and early afternoon, when he was supposed to be on the job.
Just my opinion (and ESP!).