He's a below average kick returner, the one thing he was supposed to be great at. What does taking away his WR duties do to help the team? He still is capable of having a 100+ receiving game like he had against the Bills, and by doing this you leave two very similar players, Camarillo and Bess, as the starting WR's. You shouldn't have two possession WR's starting next to each other, especially one who is fit to run the slot.
Ginn needs to stay the starting WR, no matter how much he screws up, because with those screw ups, he's bound to have moments like the one against the Jets.