It definitely IS the rubber, mine got 9 or 10 scratches the very first time I put them up. The other posts have tons of people with this problem, they had tint shops install some felt on the rubber seal, then retinted and no problems after that. Wierd part is that not everyone has had a problem. I can't find a shop by me that will install felt, they look at me like I'm crazy. I thought of maybe ordering a set of seals for the 06 thinking maybe they are improved by now, mine was bought early 05.
The guy that tinted mine here, loved the car and was excited about doing the first 05 in the area, when I brought it back with scratches I thought he was going to cry. LOL Said he would redo them for free if I can find a fix. Others have said you just need to keep the inner seals very clean and dirt free. The dirt particles don't have any place to go (two hard surfaces)