You will want to mask off the rivets around the perimeter. Maybe a couple of layers of duct tape. I would try and fashion a disk sander for the Dremel for finish work. Maybe one exists already. With a disk the size of a quarter you should be able to get close to the rivets without sanding them off. Also much faster than hand sanding.
Pete