This topic comes up pretty often. Someone high profile will criticize a false teacher and then themselves become the object of criticism for not doing it with kid gloves. When someone is a teaching false things publicly, they must be exposed publicly, and in no uncertain terms. It is one of the duty of elders to do so.
Here is direct instruction from Paul to Timothy on how to handle an elder who is sinning, 1 Timothy 5:20 (NASB Strong’s) “Those who continue in sin, rebuke in the presence of all, so that the rest also will be fearful of sinning.” This is to be done publicly.
Romans 16:17 (NASB Strong’s) “Now I urge you, brethren, keep your eye on those who cause dissensions and hindrances contrary to the teaching which you learned, and turn away from them.”
Paul even opposed Peter publicly over his false teachings and corrected him. Galatians 2:11-14 (NASB Strong’s) “But when Cephas came to Antioch, I opposed him to his face, because he stood condemned. For prior to the coming of certain men from James, he used to eat with the Gentiles; but when they came, he began to withdraw and hold himself aloof, fearing the party of the circumcision. The rest of the Jews joined him in hypocrisy, with the result that even Barnabas was carried away by their hypocrisy. But when I saw that they were not straightforward about the truth of the gospel, I said to Cephas in the presence of all, “If you, being a Jew, live like the Gentiles and not like the Jews, how is it that you compel the Gentiles to live like Jews?”
Paul on the works based teachings of the Jews,
Philippians 3:2-3 (NASB Strong’s) “Beware of the dogs, beware of the evil workers, beware of the false circumcision; for we are the true circumcision, who worship in the Spirit of God and glory in Christ Jesus and put no confidence in the flesh,”
I could go on and on about this. There are plenty of scriptures that demonstrate how to treat believers and nonbelievers who are engaged in publicly teaching falsely. I will list a few more sections of scripture at the bottom. When someone is publicly teaching, and they are doing so as a biblically unqualified person, there is no doubt how they are to be marked out, decried, debunked, and shunned until they repent. In today’s world of being able to just go down the street to the next Church, if by some chance you get put out of yours, there just isn’t any accountability. It may seem mean spirited at times, but if the motivation of the person calling out the false teacher is a love for the gospel, the Savior, and His word, then they are doing so rightly.
A false teacher who is a wolf will respond by doubling down, and going further to the left theologically. A false teacher who is an erring brother will repent. Maybe not right away while his feelings are raw and hurt, but wounds from a friend are better than the flattering kisses of an enemy.
John the Baptist publicly exposed the Sadducees and Pharisees in Matthew 3:7-8 (NASB Strong’s) “But when he saw many of the Pharisees and Sadducees coming for baptism, he said to them, “You brood of vipers, who warned you to flee from the wrath to come? Therefore bear fruit in keeping with repentance;”
Jesus warned in Matthew 7:15 (NASB Strong’s) “Beware of the false prophets, who come to you in sheep’s clothing, but inwardly are ravenous wolves.”
Again Jesus does not talk kindly and privately to these false teachers, “Matthew 12:34-36 (NASB Strong’s) You brood of vipers, how can you, being evil, speak what is good? For the mouth speaks out of that which fills the heart. The good man brings out of his good treasure what is good; and the evil man brings out of his evil treasure what is evil. But I tell you that every careless word that people speak, they shall give an accounting for it in the day of judgment.”
Jesus again publicly speaking, John 8:44 (NASB Strong’s) “You are of your father the devil, and you want to do the desires of your father. He was a murderer from the beginning, and does not stand in the truth because there is no truth in him. Whenever he speaks a lie, he speaks from his own nature, for he is a liar and the father of lies.”
Here is Dr. John Piper’s page on the topic.
Snyder's Soapbox 