St. Lawrence market has the farmers' market on the weekend, so that is worthwhile. It also makes the market very busy. The weekdays are far less crowded.
I do a big shop there on Saturday, and drop in a couple more times during the week. Because I like it, the food is great, it's on my ride home.
I had a car the first few years I lived in Toronto, and then realized I wasn't using it much, despite paying for it all the time. Now I shop for a family of four, managing just fine with either a touring bike or a backpack. The car might be important for other parts of your life, but you certainly don't need it for groceries.
Check out Bike Share. For $100 you can ride all year, and the St. Lawrence market is less than ten minutes from you by bike. The ride across Front Street is pretty easy, once you get the rhythm of the traffic.