I removed the two bolts on either side of the pump that mount it and the two lower bolts that attach the mounting plate. I didn’t notice any other bolt holes and used grease to hold the push rod in place.
There was a chunk of brass and a small black seal hanging out behind the mounting plate so that was a nice surprise.
On the front of the block lower right side, there's two bolt holes. One is a dead end, the other protrudes into the fuel pump pushrod hole. There's normally a short bolt in that hole, to keep oil from leaking out. It can be taken out so a longer bolt can be threaded in, to hold the pushrod.
Grease works fine though. But someone might have removed that bolt in the past.