Dogs certainly wouldn't pick it up - ground penetrating radar might if there was only one coffin burial above it, and if the coffin was just normal wood/veneer.
If the body was under several coffins because a lot of time had passed, or the coffin on top was lead or zinc lined, then no.
I'm a funeral director and my colleagues and I are agreed that the safest and most fool proof way to dispose of a body, would be just this scenario. It's not as simple as it sounds though because these days most graves are dug by machine and, because this is so much quicker than hand digging, they are usually dug in daylight on the day of the funeral, so a clandestine burial in the bottom of the pre-dug grave would not be possible.