Agreed. The toy car isn't fool proof evidence that this is what happened and it leaves the family wondering, certainly doesn't bring fool proof answers. There is always the chance that the toy car was left near the digger at an earlier time that day. However, on aggregate, it does increase the likelihood that a digger accident is what happened to Ben.
As for the digger having finished work or not, or having taken a break, I don't think you can rely on people's memories that much. This was something in the peripheral consciousness on that day and not what they were focused on. Memories are also suggestible things. Fact is that the digger was there and was working that day. For all we know, the digger driver was taking stuff to the fly tip that day and was going backwards and forwards leading to periods away and periods present. Then you can easily get the impression that work had stopped for the day while in fact, the digger is just up the road dropping dirt.
The digger driver sweating and shaking after the police interview is no admission of guilt though. There's toddler who vanished without a trace in an area that doesn't make the child getting lost very likely, that is too remote to make an abduction too likely - of course the digger driver must ask himself if maybe he inadvertently killed the child. That's enought to give you the shakes. Stringing this scenario along, he may not even have known then that this is what happened - always assuming it is what happened. Maybe he went back later to check the rubble at the fly tipping site.
The Needhams said that Barkas was always very helpful and 'normal' with them after Ben vanished. If he knew what he did, that takes a lot to act like that. And from what we've heard about Barkas, that would not be in character. So, no, it doesn't fully add up though 'farming accident' is the most likely scenario.