There is nothing wrong with being a college town and for Kingston that is probably the only real chance it has at warding off stagnation or decline. And I am sure that 20 years from now the city will probably have settled more comfortably into a college town/tourist city role with a somewhat healthier economy based on these functions.
The problem is what has been happening, and what will continue to happen for quite sometime is a growing division between those working at Queens, its wealthy student body, and too some extent those working and studying at RMC, and the rest of the population. Unfortunately for those not connected with the school it is a situation of insecurity over the few manufacturing jobs paying a living wage that remain, struggling to live on a low wage service sector job (and all of the negative social effects that come from this lifestyle) on top of an allready existing population segment living in poverty. And add in CFB Kingston where other than senior officials many of those families live on the borderline of poverty as well.
It is not that class divisions of this sort are new, it is just that in a city the size of Kingston they are rarely so sharp and so noticeable. As I said before in 20 years or so it will all work itself out, mainly the poor will be kicked out or just die off, but in the meantime it does create a rather unpleasent and anti-social atmosphere in many parts of the city outside the picturesque downtown area.