Correct. Well, if you swap the resistor and capacitor, on power up the + supply provides the high logic level for the reset until the cap charges and holds it low from then on.
There's an issue with the simple R-C reset circuit wrt what happens when you instantly turn off power and you have this charged capacitor. Where does the capacitor discharge to? For example, if you restore power too quickly (like in less than a second), that capacitor may not have discharged and hence there would be no reset. But in practice, and to keep things simple, this should work for digital IC chips you'd buy today due to so-called input protection diodes that quickly and safely discharges the capacitor back into the power supply.
