The timing is probably in the sear trip adjustment for the ejector, not the ejector hammer itself. This adjustment is done with a file or stone on most guns. If your gun works perfectly, you are just going to have to live with it. We all have our crosses to bear. 
