If you see a 2 yrs old child walking in the street with nobody around talking to him / holding his hand or just watching him, you have to do something. At least follow him for a little just to see if one his parents is near. It takes a few minutes and don't cost anything.
If you see a mother/father coming behind him then good. If not, the police has to be called.
And we all know bad things can happen to young kids of this age, even what would seem like the perfect, super safe residential area..
I'm not trying to attack you on what you said, I think you're right and the neighbors had this kind of thoughts when they saw him. But this should never be what anyone thinks. The world is a dangerous place and I think in 2023 everyone knows that (or atleast should know).
So I don't take this as an excuse for them.