Why are the grocery stores in New York so crappy and overpriced?

Because of the lack of space. For a store to offer good prices, they need volume--a lot of products occupying a lot of space with a lot of customers. Most supermarkets in New York (I know there are some exceptions) are too small for such volume. Also, there aren't that many of them, meaning they don't have competitive pressure to keep prices down.

They pay very high rents in Manhattan and that expense gets passed on to the customers. As was previously stated, they don't have the space to keeps a lot of items in stock so things get crowded on the shelves and the store runs out of items faster because they can only keep so much of it in stock because of that limited space. Most Manhattanites spread their shopping out...green markets for things like eggs, fruits and vegetables where they're usually fresher because they've been brought from the farm that morning. There are green markets all over the 5 boroughs and there is probably one near you. They key is to get there early so you don't have to pick from what's left at the end of the day, although quite often at the end of the day, you can negotiate the prices even lower because the sellers would rather sell items cheaper than take them back. Then cheese shops and better markets for meats and speciality items. The key is to watch the sales and buy according to what's on special and what's in season and not according to what you may want at the time. Also a lot of my friends in Manhattan go across the Queensboro to grocery shop where things are a bit cheaper and there are bigger stores. Some have shopping collectives where they pool their shopping lists and go to the Costco in Harlem and divide the bulk packs among themselves into smaller, more manageable sizes that will fit in their small cupboards and refrigerators.

If you want a box of Froot Loops, you're going to have to pay way too much for it. But for real food, there are ways and places to get things for cheap in the city if you know where to look.

They're not.

Because they can be.