In my W/Th/F school, I teach 3rd year middle schoolers, who for the most part know enough English to follow along and have conversations.

However, there are a few students in different classes who know zero English. I don't know how their level is so low when all of the other students are pretty advanced.

Question is, what should I do for these students during the classes? I hate the idea of them sitting the room mind-numbingly bored. They really need their own, simpler activity...

Doing station teaching is helpful, that way you can separate out the low level students and work with them more directly.  There's a thread on waygook about station teaching if you need tips. But personally, my classroom management skills are lacking and I find it too hard to handle most of the time.

I teach elementary but I have a few 6th graders who are illiterate, and I make sure to pair them up with higher level students and try to involve them in games using listen and repeat just to keep them interested.  I also do phonics at the beginning of every class -- very simple listening tests (I alternate between easy sounds for low-level like A/O and B/P and more difficult sounds like "IGHT"/"OUGHT" week by week) even most high level students can use the practice.  I've noticed that throwing in an easy warm up activity helps low level students engage more, sometimes even for a whole class.

If/when you do worksheets, make two versions or simply give the low level students a worksheet that is not related to class but is at their level.

If you want to put in the extra time, I have friends who have offered to teach extra after/before school phonics/basics classes and found that is really beneficial for their classrooms.  However, if you want to do this you may have to be willing for them to be "unofficial" (read: unpaid) hours.

Show some short funny videos and show some pictures giving one word answers what they are.  Find some simple power points with pictures and words.  "Chicken" "Cat" "Running" etc.  Take a break in the middle and show a youtube music video or something.  then go back to mind drubbery.  Or introduce a culture based lesson but only if the co teacher will translate.  Some kids will be surprised at the differences. Others will have no intellectual curiosity and be dumb as $h!t.  Trial and error my friend.