Routing in solaris zones
Hello
I have some questions about routing in solaris zones.
We have one Sun Fire with two physical interfaces - bge0 and bge1
#uname -a
SunOS bellatrix 5.10 Generic_118844-26 i86pc i386 i86pc
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 83.242.139.37 netmask ffffff80 broadcast 83.242.139.127
ether 0:9:3d:13:af:8b
The task is to assign on the second interface (bge1) addresses from other network (other then on bge0), and
use these addresses in several zones. For tests (see below) we are using /27 networks, but in the future
on production configuration we are planning to use /30 networks (four addresses).
Test N1
We assign address on bge1 interface and see the results
bge1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 35
inet 83.242.156.8 netmask ffffffe0 broadcast 83.242.156.31
ether 0:9:3d:13:af:8c
Routing Table: IPv4
DestinationGatewayFlags RefUseInterface
-- -- -- --
83.242.156.0 83.242.156.8 U 12 bge1
83.242.139.0 83.242.139.37U 14737 bge0
224.0.0.083.242.139.37U 10 bge0
default 83.242.139.1 UG13964
127.0.0.1127.0.0.1UH477 lo0
All looks as expected
Checking from host 83.242.156.2 (another physical machine) in 83.242.156.0/27 network
ping -s 83.242.156.8
PING 83.242.156.8: 56 data bytes
64 bytes from 83.242.156.8: icmp_seq=0. time=0.604 ms
64 bytes from 83.242.156.8: icmp_seq=1. time=0.441 ms
^C
-83.242.156.8 PING Statistics-
2 packets transmitted, 2 packets received, 0% packet loss
round-trip (ms) min/avg/max/stddev = 0.441/0.522/0.604/0.12
%traceroute -n 83.242.156.8
traceroute to 83.242.156.8 (83.242.156.8), 30 hops max, 40 byte packets
1 83.242.156.8 0.493 ms 0.366 ms 0.223 ms
All works as expected
Test N2
Very similar to test N1 but with one exception - we assign network address on alias interface, leaving bge1 without address
#ifconfig bge1 addif 83.242.156.8/27 up
#ifconfig -a
bge1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 37
inet 0.0.0.0 netmask ff000000
ether 0:9:3d:13:af:8c
bge1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 37
inet 83.242.156.8 netmask ffffffe0 broadcast 83.255.255.255
Routing Table: IPv4
DestinationGatewayFlags RefUseInterface
-- -- -- --
83.242.156.0 83.242.156.8 U 10 bge1:1
83.242.139.0 83.242.139.37U 14738 bge0
224.0.0.083.242.139.37U 10 bge0
default 83.242.139.1 UG13966
127.0.0.1127.0.0.1UH477 lo0
%ping -s 83.242.156.8
PING 83.242.156.8: 56 data bytes
64 bytes from 83.242.156.8: icmp_seq=0. time=0.610 ms
64 bytes from 83.242.156.8: icmp_seq=1. time=0.502 ms
^C
-83.242.156.8 PING Statistics-
2 packets transmitted, 2 packets received, 0% packet loss
round-trip (ms) min/avg/max/stddev = 0.502/0.556/0.610/0.076
Glue %traceroute -n 83.242.156.8
traceroute to 83.242.156.8 (83.242.156.8), 30 hops max, 40 byte packets
1 83.242.156.8 0.514 ms 0.349 ms 0.208 ms
All looks and works as expected
Test N3 (most intresting)
Very similar to test N2 but with one exception - we assign network address on alias and put this alias into zone
#zoneadm -z zone_vlan32 boot
#ifconfig bge1 addif 83.242.156.8/27 zone zone_vlan32 up
Created new logical interface bge1:1
#ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232
index 1
inet 127.0.0.1 netmask ff000000
lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232
index 1
zone mysql
inet 127.0.0.1 netmask ff000000
lo0:2: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232
index 1
zone tt
inet 127.0.0.1 netmask ff000000
lo0:3: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232
index 1
zone zone_vlan32
inet 127.0.0.1 netmask ff000000
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 83.242.139.37 netmask ffffff80 broadcast 83.242.139.127
ether 0:9:3d:13:af:8b
bge1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 37
inet 0.0.0.0 netmask ff000000
ether 0:9:3d:13:af:8c
bge1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 37
zone zone_vlan32
inet 83.242.156.8 netmask ffffffe0 broadcast 83.255.255.255
Routing Table: IPv4
DestinationGatewayFlags RefUseInterface
-- -- -- --
83.242.139.0 83.242.139.37U 14737 bge0
224.0.0.083.242.139.37U 10 bge0
default 83.242.139.1 UG13965
127.0.0.1127.0.0.1UH477 lo0
Ooops! Where the route to 83.242.156.0/27 network?
Checking from host 83.242.156.2
%ping -s 83.242.156.8
PING 83.242.156.8: 56 data bytes
64 bytes from 83.242.156.8: icmp_seq=0. time=1.01 ms
64 bytes from 83.242.156.8: icmp_seq=1. time=0.484 ms
^C
-83.242.156.8 PING Statistics-
2 packets transmitted, 2 packets received, 0% packet loss
round-trip (ms) min/avg/max/stddev = 0.484/0.747/1.01/0.37
%traceroute -n 83.242.156.8
traceroute to 83.242.156.8 (83.242.156.8), 30 hops max, 40 byte packets
1 83.242.139.37 0.986 ms 0.457 ms 0.501 ms
Ooops! Packets come back from another interface (bge0, another network!) i.e via default route
BTW, zone "zone_vlan32" have required route
#zlogin zone_vlan32
[Connected to zone 'zone_vlan32' pts/3]
Last login: Fri Jul 21 18:05:19 on pts/7
Sun Microsystems Inc.SunOS 5.10Generic January 2005
# zsh
zone-vlan32# netstat -rn
Routing Table: IPv4
DestinationGatewayFlags RefUseInterface
-- -- -- --
83.242.156.0 83.242.156.8 U 12 bge1:1
127.0.0.1127.0.0.1UH413 lo0:3
This situation can be fixed by assigning address from 83.242.156.0/27 on bge1,
but this doesn't conform initial conditions (using /30 networks)
Why when alias is inside a zone and base interface doesn't have any address, in global zone doesn't appear route to 83.242.156.0/27 netwok, like this
83.242.156.0 83.242.156.8 U 10 bge1:1
