I've explored many locations in the city where I can shop sustainably and in bulk. This page contains mini reviews and links to any major blog posts about the locations.
4th Street Food Coop (4th st, between Second and Third)
Located in the East Village. I talk about it all the time, and with good reason. Good prices and a decent selection from a place that's dedicated to organic food as well as minimizing packaging. Their aim seems to be organic first and local where possible. They're the only place I've been to so far that sells bulk spices. Their stock list is on their website - in addition to bulk food and spices, they also sell cleaning supplies like castile soap and shampoo in bulk.
I've been to the location at 75nd street and Broadway. My dominant experience with this store is that it's expensive. Granted, half the store is given over to organic everything, but even products on the non-organic side cost way too much. They do have a bulk section, but I've never bought anything from it. They won't tare customers' containers, you have to use plastic bags or plastic containers that they provide or be willing to pay a little more.
by a city non-profit, GrowNYC, the Greenmarkets bring local produce to
neighborhoods. Sometimes its organic, sometimes not, but it's always
local. In addition to produce, there's also dairy, fish, honey, and
other products, along with recycling and composting programs at some
markets. I live near a year-round market, although many farmers aren't
coming during the winter.
Live Healthfully Organic Market (4th St between First and Second)
They have a small bulk selection, along with standard organic grocery stuff. I have only bought canned goods there, which were a tad more pricey than at Whole Foods or Trader Joe's, but without the lines. They're a block away from 4th Street CoOp, so I just go there for bulk.
McNulty's Tea & Coffee Co. (Christopher St. between Hudson and Bleeker)
All they sell is tea & coffee (and some accessories) but they sell variety of both in bulk. The inexpensive teas are decent - taste-wise they're similar to bagged tea, but for a fraction of the cost. Golden Assam is my favorite from them so far. My Mom has tried their El Salvador Las Colinas (fair trade) coffee and she approves. The workers are also efficient and knowledgeable, which makes for a pleasant buying experience.
Natural Green Market (16th and Third)
I haven't actually bought anything here, I just happened across it while I was in the area. They have a nice bulk section, and a lot of other organic products, and people on their Yelp page seem supportive.
Again, I've been to the 72nd street location. Generally very well priced, but so much is wrapped in plastic there. My latest disappointment is the peanut butter - they've switched from a glass jar to plastic. I go there occasionally, to pick up meat (despite its plastic wrapping...) and canned beans. And the lines are ridiculous, almost no matter what time I go.
I've never been to the East Side location (87th and Lex), but a new location just opened up on the West Side at 81st st. and Broadway. For fancy and expensive bulk teas, this is a good place to shop. I very much enjoy their Capital of Heaven Keemun, and my best friend loves their citrus teas. They also sell pretty tins (unpackaged! but with a silicon/plastic seal), teapots/cups, and other accessories (plastic-wrapped). Just...be prepared for some sticker shock and a 2oz minimum purchase per tea. You can also get an individual serving of any tea, freshly prepared.
I've been to two locations: Upper West Side and Bowery. Both those locations have bulk. The selection at Upper West Side is much better (they have flour and sugar at UWS, along with more variety in general). The bulk is reasonably priced for the most part, but they won't tare customers' containers (I just pay a little extra). They do accept customers' containers at the meat counter, at least at UWS. I usually buy canned beans and pasta sauce there, plus other odds and ends. For the sake of your wallet and your conscience, though, stay away from their prepared food if possible. It's all expensive and wrapped in plastic.
Other options can be found at PlasticLess NYC (here and in the comments here).
Have you been to any bulk or sustainable stores in the city? Post reviews in the comments and I'll add them to the page.