I dont think that your observation is far fetched. In the end, the men are clean cut and have a uniform sort of appearance.
Though that uniform appearance is probably more commonly associated with servicemen, or in my area, some employees of large resorts, it is also associated (as you noted) with certain religious groups. Mix alcohol with members of religious groups and one can potentially get the same bad results as any other group of humans.
Then factor in that with Mennonites, Amish and Ultra Orthodox Jewish, leaving the group can have big social, family and even employment impacts. This yields some community members who are not truly religious, but remain in the group for family, social, and employment reasons. Orthodox Jews even have a Yiddish word for such members.
Needless to say, the group does not look like Orthodox Jews gone bad. But.... could they be say, Mennonites or Hutterites? Mennonite appearance can vary from "almost Amish" to "almost regular person"- but in a plain way. Or could they be say, Mormons on mission sans full uniform? Or.... less defined groups of Pentecostals or conservative Catholics?