The three men actually paid 25 shillings because that's all that reached the cashier. So they each were refunded one additional shilling plus the two the waiter pocketed. That's 25+3+2=30
Where it gets confusing is the fractional math when you say 3 men paid 10 shillings each and were owed 5/3=1.66667 in return each. Then when the waiter pockets the two shillings their return becomes 2/3=0.66667 each. In the end each man paid 9.3333 each x3=28 + the two pocketed coins =30

I'm not showing math every step of the way because time.