You're right! Not only that, trying to monitor the government's deductions, let alone get refunds of erroneous tax deductions, would be well nigh impossible!
PAYE is the best method of paying your tax. Your employer's wages dept. work it out for you, pay it direct to the taxman, AND give you a detailed statement every payday. The statement shows your your net pay and deductions details, plus details of your tax code, taxable income and tax paid to date. So that you can check their figures, if you want, and if you think it's wrong you can query it.