That one Waterfront West scheme streetcar scheme would have pretty much done this. Streetcar out of the existing 509 streetcar platform at Union, but turns right between Union and ACC, coming up in a portal in Bremner, and then running along Bremner and Fort York, and entering into a new access to the existing Exhibition GO loop (yes, there's still space with the new visitor centre) and then keep going across Dufferin, and then crossing the GO ROW onto King Street, onto Queensway, and out to Humber.
Though even the existing streetcar go Exhibition Loop to Union would serve much of Liberty Village, particularly if they were to improve the north-south connections. I expect that much if not most of the Liberty Village residents have moved in since last time that ran. It only took 18 minutes from Exhibition loop to Union.