Thanks to everyone for every comment and tip!
Mission accomplished. Hole Drilled!
After reading all comments, I realized that my project was stupidly unique, because the fragile building was already glued together and the outside walls of the building had fake raised lap boards on it. So, I found it necessary to use and amalgamation of the of the tips, blended together. JohnActions latest comments were very appreciated and helpful.
Here is a picture of the building as it started out:
Next, you can see that I taped up the wall, inside and out, with two layers of green frog tape. I used the thin frog tape because I wanted to be able to force the tape down into and between the outside clapboards. It worked out well.
Next I cut a really dry 2x2 stake to become 1 3/4 x 1 3/4 with my table saw, and put it in my vise. The building fit over it, with the wood flush against the inside wall.
My hole saw had a rather large 1/4 inch center bit, so I did as suggested and drilled a center hole in the building and wood, starting with a 1/16th bit and going up through the sizes to 1/4th. Then I inserted the 3/4 inch hole saw in my drill, and slowly ran it backwards.
It took about ten minutes, running at very low speed with just the pressure of the drill weight, to cut the hole through. I had to take 3 rests. BIG FLAG here: Since I have been using drills like this for 30 years, every time I got ready to start drilling again, my index finger instinctively pushed the direction button inward, to run the drill forward, not backward. I had to catch myself and reverse this, but once I almost forgot!
Then I carefully removed the tape. There were no cracks whatsoever, and just a little bit of white plastic "threads" hanging on in the inside of the hole. I gently removed those by hand.
The purpose of my hole is to mount my new large Motion Sensor, which is 5v to 24v DC, 5 amp, with nice 20 gauge stranded wire for input and output. The sensor fit inside perfectly, and it works great to turn on my Oil Pump and Light when my train passes by. I only paid $15 for a package of 3 of these sensors. They are motion detectors to mount on window sills. I think they are guaranteed for a full year. I just have to run the wires down through the table through and hole underneath the building, and attach them permanently together.
Thanks again to all.
Mannyrock