Honestly, King and Queen's Quay are both pretty bad locations for scheduled turnbacks (I prefer to keep the term 'short turn' for unplanned mid-route turns to adjust service, not regular planned branches of a service, but whatever).
At King, there's the cycle track issue on Adelaide, there's the massive pedestrian and car traffic at Charlotte and King making that turn difficult at times, the traffic westbound that short stretch on King can crawl, the pedestrian traffic at Spadina makes the turn difficult, and yes, the 510 turning back north delays automobile and 504 westbound traffic by a signal cycle much of the time. Also, it renders that turn basically useless for other service detours, as we saw, for instance, with TIFF service turning at Bathurst instead of at Spadina--having both the 510 and the 504 turning there, or detouring the 501 via King turning at Spadina, for whatever reason, is far too much for that intersection to handle. Some extra transit priority would go a long way towards helping, but alas. And of course this results in the massive demand at Bremner being wildly underserved, and difficult linkage to the 509 as well.
At Queen's Quay, it's honestly not all that much better. The north-south transit signal through Lakeshore is rare and brief due to not only Lakeshore but the Gardiner off-ramp being so high-volume. Again, extra transit signal priority would be a godsend, but I doubt that'll really happen. Even when eastbound lakeshore/gardiner offramp traffic has a red, there's the southbound Spadina left turn signal onto Lakeshore eastbound which requires a transit red. Once a car gets down to Queen's Quay, the east/west light is quite long, as it should be, between cars and the 509, and the streetcars need a priority signal to turn coming south on Spadina and can't turn on the general southbound green. If there's an LFLRV, or two CLRVs, servicing the eastbound stop at Queen's Quay and Spadina, they can't make their turn even on that signal or they'll block the intersection. Once they turn, offload passengers, and the driver explains to about 10 people "no, this streetcar isn't going to union, it's turning back north, please get off and wait for a streetcar that says Union Station", they need to make their left turn into the loop, which also requires a priority signal. Let's hope the loop isn't full, especially with operators stopping short of the end of the loop since it's closer to the bathroom, or else they can't enter it. Oh yeah, while they're waiting to turn into the loop, they're also blocking the 509 from continuing--again, two CLRVs or a single LFLRV and any eastbound 509s or southbound 510s can't proceed into the intersection legally. Once it turns in, let's hope the guy ahead hasn't fallen asleep, or isn't spending too long in the bathroom, at Shoppers, or at Starbucks. Then when leaving the car has to hope there are no cars blocking the loop exit by lining up at the northbound light at Lakeshore. If there are, it has to wait for them to clear, often drivers will go through on that very short green and not courteously let the streetcar go ahead of them. It has to take the turn and hope the light is still green by the time it gets onto Spadina, or it has to wait another several minutes for another green. Also, northbound and southbound cars at Bremner/Fort York and Spadina are often blocked by the massive numbers of drivers who enjoy blocking that intersection at all times of day.
So yeah, neither is perfect. I'm very much of the opinion that about half or two thirds of scheduled service should run to Queen's Quay with the rest to Union, with no regularly scheduled turnbacks at King (only rare short turns and 504/501 detour routing), as I think the issues at King and Queen's Quay are roughly equal and the demand at Bremner is high enough, but I don't think there are any signs of that happening soon. If I recall correctly, around August 31st when the LFLRVs started running, 100% of the service was running to Queen's Quay (as the route to Union was not operational yet, and I think something was going on at King that precluded using that turn) and that was fantastic for people down here...I kind of miss it. Honestly I'm even of the opinion that the 510 shouldn't run to Union, that can be served by better 509 service making the transfer relatively painless, but that's a very unpopular opinion for residents at Spadina and Bremner. Interesting to think about, if nothing else...