I think you know your child and the situation the best. One thing that helps me with decision making is to write down a list of pros and cons. While listing things, it usually becomes obvious which way I am leaning. I’ve learned to go with my gut and my mama instinct - they are usually right.
We will be withdrawing our two children at the end of this school year in May. We have had a good experience (for the most part) and really like most of the teachers and staff at the school, as do the boys, but it is not a good fit for us right now. But our situation is different and it is what will work best for US.
It seems like you already know what you would like to do, which is to go ahead and withdraw your children. I think if there are no major obstacles, like quitting a needed job, or finances, etc., then it might be in everyone’s best interests to start homeschooling now.
Big hugs and I will be praying for you!!!