For the price, I'd take a drive over to our/your local Harbor Freight and pick up one of these to do that job...
Seems like just the thing to start small, gradually enlarge. Besides, I've always been intrigued with these bits...never had an excuse to bite the bait. I'd also reduce the speed as you make the next enlarging step. Practicing on some scrap piece can also be a good teacher.
Of course, then the bit will just rattle around the workshop waiting for that next one-in-a-million opportunity!
FWIW, of course...
Lots of them on the net...lots of prices, too.
Whatever...
KD
PS...Don't think I'd try a Forstner bit...too big a biting edge. Methinks it will take a grab and your assembled wall will quickly exit stage anywhere...along with the rest of the structure.
Just a thought.