"For a person to be labeled racist they must prove racism by their actions not by their speech."

Yes. The language of words creates so many misunderstandings. But that is the world we living in these days: 'I judge you by your words, not your deeds'.
Words are just the tip of the iceberg, but the largest part of the construct lies beneath the word, our Intention. Too often, people interprete others' words according to their very own personal meaning they give these words.

The superficial ego which only sees what lies within the eye relies solely on words to draw conclusions on someone's Intention, yet it is blind to what lies without the eye. That is our empathic Heart's job.