Having now owned both systems and giving their libraries a spin it really depends on the kind of games you're into. 3DS has more worthwhile first party games and slightly better third party support. It has 3D going for it and everybody here has already pretty much talked about what the 3DS has going for it.
To me the killer app of the 3DS is Pokemon X/Y, because it has more or less been the system seller since it released. While it's probably the best Pokemon game ever, it's still Pokemon and you already know what you're expecting from it if you've played a game in the series before.
------------------------------------------
The Vita, on the other hand, is a system a lot of people seem to be underselling (or have just not used). The Vita is currently a niche device who's biggest titles are either JRPGs or Monster Hunter-like games, indie gems, or ports. It doesn't have the same kind of star power that the 3DS has but if you take a closer look or even just browse r/vita a little bit on Reddit you are bound to find games that will interest you. Hell, Vita is soon to get Borderlands 2 and Minecraft (the console versions, not that stupid pocket stuff). That has to count for something. The Vita is also a more powerful device and, while not 3D, has certain features that can improve a game
if used right. It's an underrated device that would get a lot more attention and sales if not for one glaring flaw....
The Vita uses special memory cards. While they aren't all that different from micro SDs (I think they're a tad smaller), they are different enough that micro SDs cannot be used in the system. Not only that, but Sony doesn't give out the patent to other companies, meaning that they're the only ones allowed to make these memory cards. And since they're allowed to make these memory cards themselves, they can price them
however they want. This is what I feel dooms the Vita more than anything. There may not be that many games on there but the few that are for it aren't enough to make people want to buy a Vita and then deal with these insane memory card prices (even despite the system price drop). And the system doesn't really work or save anything without a card put in. When Sony drops the price on the memory cards (AGAIN), there will probably be people more willing to buy into these cards or even the system itself.
Most people will say that the best game on the Vita is Persona 4 Golden. And while it's a good game; it's one of the few JRPGs I like; it's still just a remake of an already existing PS2 game, and doesn't really use any of the Vita's features or abilities. No, I think the Vita's flagship game is
Tearaway, a little game developed by a small team in Media Molecule, the same developers responsible for LittleBigPlanet. It's an adorable game made entirely out of crafts made from paper and uses
everything the system has to offer (yes, even the rear touchpad!). It's short but so sweet and charming that it's something every Vita owner should play, and something people interested in a Vita should be on the lookout for when playing demos on the Vita in their local Gamestop. It's a damn shame that it has so far sold poorly, partly in due to the decision to release it on the same day as the X1.