Vontae had his moments. The elite physical talent was there. I just don't think the organization ever felt he was consistent enough even though he had the interceptions.
But it's very worth noting that organization had a pretty stagnant defense for years under multiple staffs because it fixated on guys like Sean Smith, Chris Clemons, and Jimmy Wilson who couldn't get a turnover even if Mark Sanchez was delivering door to door. It had to sign Michael Thomas from another team's practice squad to get a pick to seal a comeback versus Brady, and then they taught him how to not ever get picks again too. When they drafted Will Davis and Jamar Taylor, and Davis came to camp a ballhawk and Taylor came to camp bad, they made Davis not a ballhawk and taught both of them were bad. Guys who made big plays on occasion but blew a coverage or two always got shunned for guys who were more consistently but needed virtual gifts to get a turnover and were thus as a unit entirely unable to generate turnovers.
Things seem to have changed under Flores, but there are still guys like Byron Jones who get signed to market-setting contracts despite not having a turnover in two seasons. Coaches can't help but love them some predictability.