It's really not all that complicated. Yes, you can mix O-Gauge track with O27 track, but as the old saying goes, just because you can do something doesn't mean you should. As others have pointed out, O-Gauge will give you sturdier track and more flexibility for switches, but it brings added cost.
O-Gauge no longer has the advantage of wider diameter curves. You can get 34", 42", 54", and even 72" curves in O27 so unless you're going to run $2,000 scale-size, behemoth engines, O27 will serve you well. If you've got plenty of O27 straight track, I suggest stickin with O27 UNLESS YOU WANT BETTER SWITCH SELECTION. To me, that's the big consideration for anyone starting out. Higher profile O-Gauge offers more and superior options that are independently powered (a big plus) and available in curves wider than 42". If 42" is sufficient, you can find K-Line switches in O27 profile that are independently powered with 42" diameter, but, admittedly, they're a bit hard to find on the used market.
Nothing else wrong with O27. Frankly, I find the lower profile more eye pleasing. To me, O-Gauge is too tall.
Here's are two videos of a fellow forum member's former O27 layout. He has since replaced the O27 track with Gargraves (maybe Atlas), but these videos proved to me that the track choice is a small part of the excellence of one's layout. This is one of the best I've ever seen and it was done with the cheapest track on the market.
https://www.youtube.com/watch?v=XAmTfu3f8bw
https://www.youtube.com/watch?v=-a0hYta-ZA8
