Yes that is correct. Remote control via the remote control module(not LiveConnect) will only work via the primary Kserver address. It was designed that way when it was originally created. The current workaround is to use LiveConnect when the agent is connecting via the secondary Kserver address. There are plans to change that in the future. However there is not an ETA on that.
Hope that helps.