I can't try using a brass hammer to drive the shaft out while the threads are still engaging this piece, so it needs to come off.