No, and frankly I would be concerned about that. Is he in any sort of daycare or preschool or other sort of program where he might be exposed to a pedophile?

Oh you said he's in school. I missed that part. I definitely wouldn't ignore it. If you think there is something more going on...you should investigate. But even kids that are not abused explore sexuality.

My almost six year old is not autistic and he is always talking about his genitals and has tried to grab his brothers too. He's never been abused and has mostly been in my care except for now that he's in kindergarten. Kids go through normal stages of sexual behavior too. If he does not interact with other kids and does not talk, he has no way to express himself. I'd just keep an eye on him since he's always in your care and you know he's not abused. Might just be a normal part of development. Just try to redirect him if you can. It's very easy for autistic kids to get fixated on feelings that they find pleasurable. You don't want that to become one of his things. If it becomes excessive, call the doctor.

I would further investigate the possibility of abuse just to be sure, maybe have him evaluated by a doctor or if there's a children's hospital ER or police stations have specially trained therapists who can evaluate children for sexual abuse. That said, I'm not sure what type of sexual behavior or his age, but some interest in touching their private places is a normal part of development for which they shouldn't be shamed, just educated to do it privately.
Check out freud's (I know, he's not my favorite either) phallic stage, typically it's around 3-4 but our kids are not typical.

No, he is with me when he is not in school