So I've now spent something like the last 8 hours trying to get our UTM device (which is running in an AWS instance) linked up to all our accounts and regions in AWS (VPCs). I tried several ways of getting this done and I've failed on every single one due to limitations/issues/bugs with AWS and/or UTM. Below is a recap of everything I've tried:
1) First I go and try to use the AWS VPN to link the UTM to each VPC. Well the UTM wouldn't import the configuration and would error with "No VPC connection found for local system" when trying to import the Astaro config provided by AWS. Ended up finding this post which says to modify the config and replace the external IP of the UTM instance with the internal IP. After fixing that the connection comes up and works great.
2) Now I go and try to set up a second VPN connection. But as soon as you try to set up a 'customer gateway' in another VPC, AWS squawks and says "Conflict among chosen gateway IP addresses". This led me to this, this, and this on the AWS forums which basically say that you cannot use the same IP for more than 1 'customer gateway.
3) So I then tried adding multiple IPs to the UTM instance (which you can only do if you run it inside a VPC). This errord with "The cable modem interface object 'Internal' is write protected". I then dug up this post on the forums here which says you cannot add multiple IPs to the primary interface of the UTM AWS AMI.
4) Next I tried adding another interface per the recommendation of the previous forum post. The first issue I hit here was that the UTM wouldn't pick up the interface. I logged into the UTM via root ssh and confirmed the interface was present, it just wasn't getting picked up in the web UI. As a last resort I decided to eat a little downtime and try a reboot. This finally got the interface to be picked up. So now I have another interface with it's own IP, a secondary IP added to the main interface in AWS (but added it to the second interface in the UTM), and the AWS ElasticIP associated with that secondary IP.
5) Then I try and import the second AWS VPN configuration again and it says "VPC import successful". Yay!. I go to the VPN status page in UTM, the VPN connection isn't listed. I try importing again, I get another success message, but nope, the connection is still missing. WTF? After banging my head against the wall I delete the first AWS VPN configuration and try importing the second config again. Now I get "No VPC connection found for local system". Well, at least I have an error message now. So I dig up this post which says that you can't use secondary IPs for VPN connections.
6) So I back out the multiple IPs on a single AWS interface and instead try to add more interfaces to the instance with one IP per interface. Well apparently AWS only lets you have 3 interfaces per instance, so this route seems doomed to fail right there. But I continue anyway trying to get at least some of the links up.
7) Now the interface is up, and I try importing the AWS VPN config once more. It says it imports fine again, I check the status page and the link is down. Look in the error log and I see "ERROR: "S_vpn-staging [1]" #91: sendto on eth2 to 204.246.163.76:500 failed in main_outI1. Errno 1: Operation not permitted". *sigh*
And now I'm stuck.
As you can see from the huge complicated list of issues, this task has become an utter nightmare. It feels like this stuff was barely tested.
P.S. As I was composing this post, I found that there is a bug in the forum as well. Clicking the font color button generates invalid markup. [COLOR="rgb(139, 0, 0)"]Like this[/COLOR].
Enough bugs already! :mad:
1) First I go and try to use the AWS VPN to link the UTM to each VPC. Well the UTM wouldn't import the configuration and would error with "No VPC connection found for local system" when trying to import the Astaro config provided by AWS. Ended up finding this post which says to modify the config and replace the external IP of the UTM instance with the internal IP. After fixing that the connection comes up and works great.
2) Now I go and try to set up a second VPN connection. But as soon as you try to set up a 'customer gateway' in another VPC, AWS squawks and says "Conflict among chosen gateway IP addresses". This led me to this, this, and this on the AWS forums which basically say that you cannot use the same IP for more than 1 'customer gateway.
3) So I then tried adding multiple IPs to the UTM instance (which you can only do if you run it inside a VPC). This errord with "The cable modem interface object 'Internal' is write protected". I then dug up this post on the forums here which says you cannot add multiple IPs to the primary interface of the UTM AWS AMI.
4) Next I tried adding another interface per the recommendation of the previous forum post. The first issue I hit here was that the UTM wouldn't pick up the interface. I logged into the UTM via root ssh and confirmed the interface was present, it just wasn't getting picked up in the web UI. As a last resort I decided to eat a little downtime and try a reboot. This finally got the interface to be picked up. So now I have another interface with it's own IP, a secondary IP added to the main interface in AWS (but added it to the second interface in the UTM), and the AWS ElasticIP associated with that secondary IP.
5) Then I try and import the second AWS VPN configuration again and it says "VPC import successful". Yay!. I go to the VPN status page in UTM, the VPN connection isn't listed. I try importing again, I get another success message, but nope, the connection is still missing. WTF? After banging my head against the wall I delete the first AWS VPN configuration and try importing the second config again. Now I get "No VPC connection found for local system". Well, at least I have an error message now. So I dig up this post which says that you can't use secondary IPs for VPN connections.
6) So I back out the multiple IPs on a single AWS interface and instead try to add more interfaces to the instance with one IP per interface. Well apparently AWS only lets you have 3 interfaces per instance, so this route seems doomed to fail right there. But I continue anyway trying to get at least some of the links up.
7) Now the interface is up, and I try importing the AWS VPN config once more. It says it imports fine again, I check the status page and the link is down. Look in the error log and I see "ERROR: "S_vpn-staging [1]" #91: sendto on eth2 to 204.246.163.76:500 failed in main_outI1. Errno 1: Operation not permitted". *sigh*
And now I'm stuck.
As you can see from the huge complicated list of issues, this task has become an utter nightmare. It feels like this stuff was barely tested.
P.S. As I was composing this post, I found that there is a bug in the forum as well. Clicking the font color button generates invalid markup. [COLOR="rgb(139, 0, 0)"]Like this[/COLOR].
Enough bugs already! :mad: