VPN S2S secondary interface

I am trying to set all Site to Site IPSec VPN traffic to use a secondary ISP which is not on the primary interface.

When selecting the primary external interface for "Local Interface" the connection is established, when selecting ISP2 interface as the "Local Interface" the connection fails

Is there something I am missing that needs to be configured?

Thanks in advance Bob