Then just have Go take over the stops as an express regional service, and knock it down to a stop in Hamilton and one in Toronto. In fact, both Niagara Falls-es could be served by a station across the river, which is US soil, but has an easy customs area to transfer over to the Canadian Niagara Falls, including express Go service, if such a station would even belong on a HSR routing.
In reality, Toronto-New York should be a transferless route. There will soon be demand for an Express Go service along the Lakeshore corridor, and with that there will both be little demand for a Toronto-Hamilton trip, and it could get rid of all the smaller stops that VIA uses now.