I feel it's a deep poem that can bring more interpretations, but that's what I got.

The poem talks about time. Of the temporal relationship with life. Of the eternal relationship with God or the divine.
The relationship between the "I" and the "you" is distant, but influential. The distance, time and lack of interest of that "you" towards the lyrical voice creates imbalance, dispersion, lack of love, annihilation. The annulment of himself for lack of reasons to be who he is. The whole poem expresses a claim. For me, that "ghost" can be God or maybe the human conscience, that complement of oneself that accompanies us. Congratulations for a great job, @mamadini.