You might try FLEX-I-FILE flexible sanding (polishing) sticks available at hobby shops or online. Model airplane, model car and model railroad hobbyists use them. Four grades available, the extra fine might be all you need for your rivets. These are about four inches long and 3/8 inch wide coming to a near point on one end, sanding or polishing surface on both sides.
I use these extensively in progressively finer light sanding and polishing of my non-anodized alloy bike parts before going on to Simichrome or Mother's. Being a bit flexible they are great for getting into small spots or working on curved parts.
Paul Lee
High Point, NC USA