Notwithstanding the clusterf**k this looks like it may turn into, the truth is that the Government can and does recover monies owed by former CF members. The Financial Administration Act requires it. Simply writing off a debt of this nature is well nigh impossible (there are exceptions of course, but the easiest way is to die without an estate).
Recovering overpayments from annuitants is simple. If after the Government sends you a number of polite letters informing you of the debt, you have not repaid the amount, they simply take it off your pension cheque.
Recovering from non-annuitants is a little more difficult, but not impossible. After asking you nicely, your case will be sent to a collection agency. If that doesn't work, after one year, they take it off your income tax.
The lesson here is that unless you disappear from society completely, the Government will get its money (provided they find the error within six years - in that case, you get to keep it).