Time travel is a cool story device and I love to read stories or watch movies or TV show that contain Time travel elements.
But I don't think we have to worry about the actual invention of such a technology, since scientist agree that the energy needed to transport any matter through time would be unimaginable high and unobtainable.
So we are safe from anyone using that power against us.
But still it's cool to think about the "what if?"