Well, they remember until they forget - the whole stack, not just few nuts. :)
There is also a "subculture" of people that trains squirrels to harvest hazelnuts for them; and then steal them from poor creatures. :)