I'm guessing that the neighbor might have been correct. How would BD know if she was on the phone inside or outside when she was not there? We know what we would like to believe our kids do when we are gone, but now that mine are grown, turns out that what I 'thought' they were doing while I was at work was not always the case.
The neighbor had no reason to lie about it, I think. He may creep us out, but neighbors who have a vantage point usually do see more than we realize.
I had a creepy neighbor once (young mother with a new baby at home) who one day said to me, "I see you have a dog crate now in the family room. Are you getting a puppy?" Her house was on the opposite side of my driveway and the only way she would have known this was if she was looking through my patio door that faced her house. She also commented on the fact that I was exercising more than she was because, apparently, she watched me while I was Sweating to the Oldies with Richard Simmons on my TV. Needless to say, we shut the blinds a lot more after that.