I agree, you don't need to understand the deeper concepts of a piece to appreciate it. Indeed I think that art is taken in and processed by our senses and enjoyed at that raw level before any intellectualising of it has the time to happen.
I also agree that further engagement of a work on an intellectual level i.e. thinking about the context that it was made, the themes of the work, the artists intentions etc can enhance our appreciation of art immensely.