I don't have a problem speaking my mind to people anymore. I used to but thankfully I learned that you have to just say it like it is or they will continue to insidiously bother you and claim otherwise.
"O you who have believed, respond to Allah and to the Messenger when he calls you to that which gives you life. And know that Allah intervenes between a man and his heart and that to Him you will be gathered." ~ Quran 8:24