This first entry will look at the low level creation and configuration of vNICs, vIPs, vLANs, BONDs and how we can leverage scripting and the dcli command to configure all Exalogic Compute Nodes at the same time. At this point in time the scripts with documented within this page are designed from Oracle Enterprise Linux but can be tweaked slightly for a Solaris Exalogic installation. In addition we will focus on the external facing 10Gb network because the the 1Gb is management only and should be on a private network whilst the internal InfiniBand network will not be directly accessible to the outside world.
vNICs
Virtual Network Interface Cards (vNICs) emulates a NIC for a given Compute Node within the Exalogic Rack. As part of the installation and configuration the Oracle Engineer will create a number of these, based on a predefined algorithm, to allow 10Gb connection through the switches within the rack. Internally this is implemented over the InfiniBand Network and converted to 10Gb at the switch.As mentioned the Engineer will create the vNICs with MAC addresses based on a predefined algorithm that takes the GUID for each Compute Nodes Network card and a value indicating which exernal Ethernet port we will be connecting it to. In reality the only requirement is that the MAC addresses be unique. Given the fact that hand cranking these for a full, 30 node, rack is time consuming and error prone I created a number of scripts that generate a simple script file containing all the required createvnic commands.
#!/bin/sh # "el2bcn15 192.168.10.67 VNIC createvnic 0A-ETH-1 -guid 00:21:28:00:01:a0:f5:fa -mac a0:f5:fa:10:00:67 -pkey default # "el2bcn16 192.168.10.68 VNIC createvnic 0A-ETH-1 -guid 00:21:28:00:01:a0:fb:4a -mac a0:fb:4a:10:00:68 -pkey default # "el2bcn08 192.168.10.58 VNIC createvnic 0A-ETH-1 -guid 00:21:28:00:01:a0:d5:85 -mac a0:d5:85:10:00:58 -pkey default # "el2bcn09 192.168.10.59 VNIC createvnic 0A-ETH-1 -guid 00:21:28:00:01:a0:d6:fe -mac a0:d6:fe:10:00:59 -pkey default # "el2bcn12 192.168.10.62 VNIC createvnic 0A-ETH-1 -guid 00:21:28:00:01:a0:d5:6e -mac a0:d5:6e:10:00:62 -pkey default # "el2bcn11 192.168.10.61 VNIC createvnic 0A-ETH-1 -guid 00:21:28:00:01:a0:d7:ea -mac a0:d7:ea:10:00:61 -pkey default # "el2bcn14 192.168.10.64 VNIC createvnic 0A-ETH-1 -guid 00:21:28:00:01:a1:0c:0e -mac a0:0c:0e:10:00:64 -pkey default # "el2bcn13 192.168.10.63 VNICOnce generated this file can be executed on the switches to create the actual vNICs that can then be used by the compute node to access / be accessed from external systems using the 10Gb network. Before this can be done we will need to create / configure the Ethernet and Bonds on each compute node as described in the next section.
The two scripts below, genVNICsFromLinkListUp.sh and genVNICsFromIBNetDiscover.sh, use the existing Exalogic scripts linklistup and inbnetdiscover to obtain information about the Exalogic network configuration. If you already have a 10Gb cable plugged into the Exalogic switch then using the genVNICsFromLinkListUp.sh will identify this and create the script file based on the active link by calling the genVNICsFromIBNetDiscover.sh passing the information. If on the other hand you need to create the vNICs before the cable has been connected then you can use the genVNICsFromIBNetDiscover.sh directly passing the Ethernet port id that will be used.
genVNICsFromLinkListUp.sh
1 #!/bin/sh 2 3 ################################################################################ 4 # 5 # Exalogic EL X2-2 1.0 (Linux x86-64) Configuration Script. 6 # 7 # HEADER START 8 # 9 # THIS SCRIPT IS PROVIDED ON AN �AS IS� BASIS, WITHOUT WARRANTY OF ANY KIND, 10 # EITHER EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES THAT 11 # THE COVERED SCRIPT IS FREE OF DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR 12 # PURPOSE OR NON-INFRINGING. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE 13 # OF THE COVERED SOFTWARE IS WITH YOU. SHOULD ANY COVERED SOFTWARE PROVE 14 # DEFECTIVE IN ANY RESPECT, YOU (NOT THE INITIAL DEVELOPER OR ANY OTHER 15 # CONTRIBUTOR) ASSUME THE COST OF ANY NECESSARY SERVICING, REPAIR OR CORRECTION. 16 # NO USE OF ANY COVERED SOFTWARE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS 17 # DISCLAIMER. 18 # 19 # When distributing this Code, include this HEADER in each file. 20 # If applicable, add the following below this this HEADER, with the fields 21 # enclosed by brackets "[]" replaced with your own identifying information: 22 # Portions Copyright [yyyy] [name of copyright owner] 23 # 24 # HEADER END 25 # 26 # 27 # Copyright 2011 Andrew Hopkinson, Oracle Corporation UK Ltd. 28 # 29 ################################################################################ 30 31 32 ################################################################################ 33 # 34 # genVNICsFromListLinkUp.sh 35 # ========================= 36 # 37 # Version : 1.0.1 38 # 39 # This script does not directly affect the system it is running rather it calls 40 # the genVNICsFromIBNetDiscover.sh script for each ETH port it finds which is up. 41 # On execution of the genVNICsFromIBNetDiscover.sh script it will pass the ETH 42 # port and output from the ibnetdiscover. 43 # 44 # Parameters 45 # 46 # -f : Specifies a file that contain the output from the listlinkup command 47 # : executed on the switch. If no file is provided the script will execute 48 # : the listlinkup command to generate the file. 49 # -i : If this script is not being run on a switch or you already have an 50 # : output from ibnetdiscover that you want to use then it can be specified 51 # : with this flag. If the flag is not present the script will execute 52 # : ibnetdiscover to generate the file. 53 # -v : If the script is not being run on a switch then you must supply the 54 # : output from showvnics. If the flag is not present then the script will 55 # : execute showvnics and write the content to a temporary file. 56 # 57 # Version History 58 # 1.0.0 - Initial Realease 59 # 1.0.1 - Modified the generation of the delete VNICs script so that it is 60 # generated based on the output from showvnics 61 # 62 ################################################################################ 63 64 65 IBNETDISCOVER= 66 LISTLINKUPOUT= 67 SHOWVNICSFILE= 68 69 while [ $# -gt 0 ] 70 do 71 case "$1" in 72 -f) LISTLINKUPOUT="$2"; shift;; 73 -i) IBNETDISCOVER="$2"; shift;; 74 -v) SHOWVNICSFILE="$2"; shift;; 75 *) echo ""; echo >&2 \ 76 "usage: $0 [-f listlinkup output] [-i ibnetdiscover output] [-v showvnics output] " 77 echo""; exit 1;; 78 *) break;; 79 esac 80 shift 81 done 82 83 # Generate ibnetdiscover 84 if test "$IBNETDISCOVER" = "" 85 then 86 echo "Generating ibnetdiscover" 87 echo "" 88 IBNETDISCOVER=ibnetdiscover.out 89 ibnetdiscover > $IBNETDISCOVER 90 if [ "$?" -ne "0" ] 91 then 92 echo "" 93 echo "Error: Generating ibnetdiscover output only works on the switch" 94 echo "" 95 exit 1 96 fi 97 fi 98 99 # Generate showvnics 100 if test "$SHOWVNICSFILE" = "" 101 then 102 echo "Generating showvnics" 103 echo "" 104 SHOWVNICSFILE=showvnics.out 105 showvnics > $SHOWVNICSFILE 106 if [ "$?" -ne "0" ] 107 then 108 echo "" 109 echo "Error: Generating showvnics output only works on the switch" 110 echo "" 111 exit 1 112 fi 113 fi 114 115 # Generate listlinkup 116 if test "$LISTLINKUPOUT" = "" 117 then 118 echo "Generating listlinkup" 119 echo "" 120 LISTLINKUPOUT=listlinkup.out 121 listlinkup > $LISTLINKUPOUT 122 if [ "$?" -ne "0" ] 123 then 124 echo "" 125 echo "Error: Generating listlinkup output only works on the switch" 126 echo "" 127 exit 1 128 fi 129 fi 130 131 TEMPBRIDGEFILE=.upEthBridges 132 133 grep Bridge $LISTLINKUPOUT | grep "ETH-" | grep "up" > $TEMPBRIDGEFILE 134 135 while read line 136 do 137 i=0 138 for el in $line 139 do 140 i=`expr $i + 1` 141 # echo "$i = $el" 142 if [ $i -eq 3 ] 143 then 144 # echo "Port $el" 145 ./genVNICsFromIBNetDiscover.sh -f $IBNETDISCOVER -p $el -v $SHOWVNICSFILE 146 break 147 fi 148 done 149 done < $TEMPBRIDGEFILE 150 151 # Cleanup 152 153 rm $TEMPBRIDGEFILE 154 155
genVNICsFromIBNetDiscover.sh
1 #!/bin/sh 2 3 ################################################################################ 4 # 5 # Exalogic EL X2-2 1.0 (Linux x86-64) Configuration Script. 6 # 7 # HEADER START 8 # 9 # THIS SCRIPT IS PROVIDED ON AN �AS IS� BASIS, WITHOUT WARRANTY OF ANY KIND, 10 # EITHER EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES THAT 11 # THE COVERED SCRIPT IS FREE OF DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR 12 # PURPOSE OR NON-INFRINGING. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE 13 # OF THE COVERED SOFTWARE IS WITH YOU. SHOULD ANY COVERED SOFTWARE PROVE 14 # DEFECTIVE IN ANY RESPECT, YOU (NOT THE INITIAL DEVELOPER OR ANY OTHER 15 # CONTRIBUTOR) ASSUME THE COST OF ANY NECESSARY SERVICING, REPAIR OR CORRECTION. 16 # NO USE OF ANY COVERED SOFTWARE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS 17 # DISCLAIMER. 18 # 19 # When distributing this Code, include this HEADER in each file. 20 # If applicable, add the following below this this HEADER, with the fields 21 # enclosed by brackets "[]" replaced with your own identifying information: 22 # Portions Copyright [yyyy] [name of copyright owner] 23 # 24 # HEADER END 25 # 26 # 27 # Copyright 2011 Andrew Hopkinson, Oracle Corporation UK Ltd. 28 # 29 ################################################################################ 30 31 32 ################################################################################ 33 # 34 # genVNICsFromIBNetDiscover.sh 35 # ============================ 36 # 37 # Version : 1.0.2 38 # 39 # This does not affect the system directly rather it build a number of other 40 # scripts that the user can execute to modify the system. The scripts it 41 # created are in in the format : 42 # create-- 43 # 44 # This file contain a series of createvnic commands similar to those below 45 # 46 # #!/bin/sh 47 # # "el01cn15 192.168.10.17 VNIC 48 # createvnic 0A-ETH-1 -guid 00:21:28:00:01:a0:f5:fa -mac a0:f5:fa:10:00:17 -pkey default 49 # # "el01cn16 192.168.10.18 VNIC 50 # createvnic 0A-ETH-1 -guid 00:21:28:00:01:a0:fb:4a -mac a0:fb:4a:10:00:18 -pkey default 51 # # "el01cn08 192.168.10.8 VNIC 52 # createvnic 0A-ETH-1 -guid 00:21:28:00:01:a0:d5:85 -mac a0:d5:85:10:00:08 -pkey default 53 # 54 # Once created the user can then execute this file on the switch or edit it as 55 # required. 56 # 57 # Parameters 58 # 59 # -a : Specifies is the script is to generate create-$swname-$ETHPORT-VNICs.sh 60 # : files for all switches that can be identified from the ibnetdiscover 61 # : output. By default this is false and this script will only generate 62 # : output for the first switch found that is the one it is being run on. 63 # -f : If this script is not being run on a switch or you already have an 64 # : output from ibnetdiscover that you want to use then it can be specified 65 # : with this flag. If the flag is not present the script will execute 66 # : ibnetdiscover to generate the file. 67 # -p : Defines the ETH Port for which thes create..... script will be generated 68 # : by default we assume thise is 0A-ETH-1. 69 # -v : If the script is not being run on a switch then you must supply the 70 # : output from showvnics. If the flag is not present then the script will 71 # : execute showvnics and write the content to a temporary file. 72 # -c : Flag to indicate that the /conf/bx.conf should be cleared. 73 # 74 # When called from the genVNICsFromListLookUp.sh script -a will be ommited 75 # but -f and -p will be specified. 76 # 77 # Version History 78 # 1.0.0 - Initial Realease 79 # 1.0.1 - Modified the generation of the delete VNICs script so that it is 80 # generated based on the output from showvnics 81 # 1.0.2 - Added " service bxm restart" to the end of the deletevnincs generated 82 # file. 83 # 84 ################################################################################ 85 86 87 IBNETDISCOVER= 88 ETHPORT=0A-ETH-1 89 PORTNO=00 90 ALLSWITCHES=false 91 SHOWVNICSFILE= 92 CLEARBXCONF=false 93 94 while [ $# -gt 0 ] 95 do 96 case "$1" in 97 -a) ALLSWITCHES=true;; 98 -c) CLEARBXCONF=true;; 99 -f) IBNETDISCOVER="$2"; shift;; 100 -p) ETHPORT="$2"; shift;; 101 -v) SHOWVNICSFILE="$2"; shift;; 102 *) echo ""; echo >&2 \ 103 "usage: $0 [-f ibnetdiscover output] [-p Ethernet Port (eg 0A-ETH-1)] [-a generate for all visible switches] [-v showvnics output] [-c clear /conf/bx.conf]" 104 echo""; exit 1;; 105 *) break;; 106 esac 107 shift 108 done 109 # "usage: $0 [-f ibnetdiscover output] [-p Ethernet Port (eg 0A-ETH-1)] [-n Machine Name Prefix (eg el01)]" 110 111 # Generate ibnetdiscover 112 if test "$IBNETDISCOVER" = "" 113 then 114 echo "Generating ibnetdiscover" 115 echo "" 116 IBNETDISCOVER=ibnetdiscover.out 117 ibnetdiscover > $IBNETDISCOVER 118 if [ "$?" -ne "0" ] 119 then 120 echo "" 121 echo "Error: Generating ibnetdiscover output only works on the switch" 122 echo "" 123 exit 1 124 fi 125 fi 126 127 # Generate showvnics 128 if test "$SHOWVNICSFILE" = "" 129 then 130 echo "Generating showvnics" 131 echo "" 132 SHOWVNICSFILE=showvnics.out 133 showvnics > $SHOWVNICSFILE 134 if [ "$?" -ne "0" ] 135 then 136 echo "" 137 echo "Error: Generating showvnics output only works on the switch" 138 echo "" 139 exit 1 140 fi 141 fi 142 143 144 if test "$ETHPORT" = "0A-ETH-1" 145 then 146 PORTNO=10 147 fi 148 if test "$ETHPORT" = "0A-ETH-2" 149 then 150 PORTNO=20 151 fi 152 if test "$ETHPORT" = "0A-ETH-3" 153 then 154 PORTNO=30 155 fi 156 if test "$ETHPORT" = "0A-ETH-4" 157 then 158 PORTNO=40 159 fi 160 if test "$ETHPORT" = "1A-ETH-1" 161 then 162 PORTNO=50 163 fi 164 if test "$ETHPORT" = "1A-ETH-2" 165 then 166 PORTNO=60 167 fi 168 if test "$ETHPORT" = "1A-ETH-3" 169 then 170 PORTNO=70 171 fi 172 if test "$ETHPORT" = "1A-ETH-4" 173 then 174 PORTNO=80 175 fi 176 177 DELETEVNICSFILE= 178 CREATEVNICSFILE= 179 180 # Process file 181 182 switchcount=0 183 ln=0 184 vn=0 185 while read line 186 do 187 ln=`expr $ln + 1` 188 # Check for IB Switch 189 i=0 190 createfiles=false 191 swname=Unknown 192 for el in $line 193 do 194 i=`expr $i + 1` 195 if [ $i -eq 1 ] 196 then 197 if test "$el" != "Switch" 198 then 199 break 200 fi 201 fi 202 if [ $i -eq 6 ] 203 then 204 if test "$el" != "IB" 205 then 206 break 207 fi 208 fi 209 if [ $i -eq 7 ] 210 then 211 if test "$el" != "QDR" 212 then 213 break 214 fi 215 fi 216 if [ $i -eq 10 ] 217 then 218 switchname=$el 219 if test "$el" != "localhost" 220 then 221 swname=$el 222 createfiles=true 223 break 224 fi 225 fi 226 if [ $i -eq 11 ] 227 then 228 swname=$el 229 createfiles=true 230 break 231 fi 232 # echo "$i el = $el" 233 done 234 235 # Check if we need to create files 236 if test "$createfiles" = "true" 237 then 238 createfiles=false 239 switchcount=`expr $switchcount + 1` 240 241 if [ $switchcount -eq 2 ] 242 then 243 if test "$ALLSWITCHES" != "true" 244 then 245 break 246 fi 247 fi 248 249 CREATEVNICSFILE=create-$swname-$ETHPORT-VNICs.sh 250 echo "#!/bin/sh" > $CREATEVNICSFILE 251 chmod +x $CREATEVNICSFILE 252 fi 253 254 # Check for Connection 255 i=0 256 for el in $line 257 do 258 i=`expr $i + 1` 259 if [ $i -eq 2 ] 260 then 261 sc=`expr substr $el 2 1` 262 if test "$sc" != "H" 263 then 264 break 265 fi 266 267 oct1=`expr substr $el 25 2` 268 oct2=`expr substr $el 27 2` 269 oct3=`expr substr $el 29 2` 270 oct4=`expr substr $el 31 2` 271 oct5=`expr substr $el 33 2` 272 oct6=`expr substr $el 35 2` 273 oct7=`expr substr $el 37 2` 274 275 guid=00:$oct1:$oct2:$oct3:$oct4:$oct5:$oct6:$oct7 276 fi 277 if [ $i -eq 4 ] 278 then 279 cn=$el 280 fi 281 if [ $i -eq 5 ] 282 then 283 if test "$el" != "EL-C" 284 then 285 break 286 fi 287 fi 288 if [ $i -eq 6 ] 289 then 290 ip=$el 291 len=`expr length $el` 292 start=`expr $len - 1` 293 mac3=`expr substr $el $start 2` 294 if test "`expr substr $mac3 1 1`" = "." 295 then 296 start=`expr $start + 1` 297 mac3=0`expr substr $el $start 1` 298 fi 299 300 mac=a0:$oct6:$oct7:$PORTNO:00:$mac3 301 302 # Write to file 303 echo "# $cn $ip VNIC" >> $CREATEVNICSFILE 304 echo "createvnic $ETHPORT -guid $guid -mac $mac -pkey default" >> $CREATEVNICSFILE 305 306 fi 307 # echo "$i el = $el" 308 done 309 310 done < $IBNETDISCOVER 311 312 313 echo "Generated Switch Files" 314 315 # Generate DELETEVNICS File 316 DELETEVNICSFILE=delete-$ETHPORT-VNICs.sh 317 echo "#!/bin/sh" > $DELETEVNICSFILE 318 chmod +x $DELETEVNICSFILE 319 320 SHOWVNICSFILEGREP=$SHOWVNICSFILE.grep 321 grep $ETHPORT $SHOWVNICSFILE > $SHOWVNICSFILEGREP 322 while read line 323 do 324 i=0 325 for el in $line 326 do 327 i=`expr $i + 1` 328 if [ $i -eq 1 ] 329 then 330 vn=$el 331 echo "deletevnic $ETHPORT $vn" >> $DELETEVNICSFILE 332 break 333 fi 334 done 335 done < $SHOWVNICSFILEGREP 336 337 rm $SHOWVNICSFILEGREP 338 339 if test "$CLEARBXCONF" = "true" 340 then 341 echo "echo > /conf/bx.conf" >> $DELETEVNICSFILE 342 fi 343 344 echo "service bxm restart" >> $DELETEVNICSFILE 345 346 echo "Generated Delete VNICS Files" 347 348-VNICs.sh
Bonds and Eth Config
Once the vNICs have been created we need to configure each of the Compute Nodes to connect to each of the switches and to do this we will create two Ethernet configuration files, by convention named ifcfg-eth4 and ifcfg-eth5, which are the physical connections to the ports. In addition we will create a bonded port, by convention name ifcfg-bond1, which it the logical port that the Compute Node will connect through. The creation of the bonded (bond1) network provides network resilience and fail-over. The contents of these files are similar to the following :Ifcfg-eth4
DEVICE=eth4 BOOTPROTO=none ONBOOT=yes HWADDR=A0:F9:1E:50:00:32 MASTER=bond1 SLAVE=yes
Ifcfg-eth5
DEVICE=eth5 BOOTPROTO=none ONBOOT=yes HWADDR=A0:F9:1D:50:00:32 MASTER=bond1 SLAVE=yes
Ifcfg-bond1
DEVICE=bond1 IPADDR=10.131.12.94 NETMASK=255.255.255.0 BOOTPROTO=none USERCTL=no TYPE=Ethernet ONBOOT=yes IPV6INIT=no BONDING_OPTS="mode=active-backup miimon=100 downdelay=5000 updelay=5000" GATEWAY=10.131.12.1In addition to allow default routing over the 10Gb Network we will need to modify the network conguration to specify the 10Gb network as the default routing option.
Again I have created some simple scripts to simplify the work and running the genCNNetFiles.sh will generated an new script that uses dcli to distribute the genIfcfgFiles.sh and execute it using ssh. This will generate the appropriate ifcfg files based on the outputs from the previous create vNICs execution.
genCNNetFiles.sh
1 #!/bin/sh 2 3 ################################################################################ 4 # 5 # Exalogic EL X2-2 1.0 (Linux x86-64) Configuration Script. 6 # 7 # HEADER START 8 # 9 # THIS SCRIPT IS PROVIDED ON AN �AS IS� BASIS, WITHOUT WARRANTY OF ANY KIND, 10 # EITHER EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES THAT 11 # THE COVERED SCRIPT IS FREE OF DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR 12 # PURPOSE OR NON-INFRINGING. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE 13 # OF THE COVERED SOFTWARE IS WITH YOU. SHOULD ANY COVERED SOFTWARE PROVE 14 # DEFECTIVE IN ANY RESPECT, YOU (NOT THE INITIAL DEVELOPER OR ANY OTHER 15 # CONTRIBUTOR) ASSUME THE COST OF ANY NECESSARY SERVICING, REPAIR OR CORRECTION. 16 # NO USE OF ANY COVERED SOFTWARE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS 17 # DISCLAIMER. 18 # 19 # When distributing this Code, include this HEADER in each file. 20 # If applicable, add the following below this this HEADER, with the fields 21 # enclosed by brackets "[]" replaced with your own identifying information: 22 # Portions Copyright [yyyy] [name of copyright owner] 23 # 24 # HEADER END 25 # 26 # 27 # Copyright 2011 Andrew Hopkinson, Oracle Corporation UK Ltd. 28 # 29 ################################################################################ 30 31 32 ################################################################################ 33 # 34 # genCNNetFiles.sh 35 # ================ 36 # 37 # Version : 1.0.2 38 # 39 # This does not affect the system directly rather it build a number of other 40 # scripts that the user can execute to modify the system. The three scripts it 41 # creates are: 42 # createAllIfCfg.sh 43 # setDefaultRouting.sh 44 # add10gIPToHosts.sh 45 # In addition it relies on the existance of the genIfcfgFiles.sh within the same 46 # directory. 47 # 48 # Parameters 49 # 50 # -d : Home directory of the dcli command 51 # -n : Node list to need processed 52 # -ip : A file containing a list (one per line) IP Address that will be 53 # : allocated to the nodes. This allows for non sequential IPs 54 # -nm : The Bond1 Network Mask 55 # -gw : The Gateway to be used by the 10 Gb bond1 conection. 56 # -vnic1 : File containing the output for showvnics on Gateway Switch 1 57 # : The contents will be paired with the entry for -vnics2 to 58 # : create the ifcfg-eth4 file. (Compute Nodes 1-16) 59 # -vnic2 : File containing the output for showvnics on Gateway Switch 2 60 # : The contents will be paired with the entry for -vnics1 to 61 # : create the ifcfg-eth5 file. (Compute Nodes 1-16) 62 # -vnic3 : File containing the output for showvnics on Gateway Switch 3 63 # : The contents will be paired with the entry for -vnics4 to 64 # : create the ifcfg-eth4 file. (Compute Nodes 17-30) 65 # -vnic4 : File containing the output for showvnics on Gateway Switch 4 66 # : The contents will be paired with the entry for -vnics3 to 67 # : create the ifcfg-eth5 file. (Compute Nodes 17-30) 68 # 69 # createAllIfCfg.sh 70 # ----------------- 71 # 72 # This generated script will use dcli to copy the genIfcfgFiles.sh to all nodes 73 # and then execute this script using a series of ssh commands. These ssh commands 74 # commands are built within the main script and pass in compute node specific 75 # information (see genIfcfgFiles.sh header for information). On completion of 76 # the script the genIfcfgFiles.sh will be removed from all compute nodes. 77 # 78 # setDefaultRouting.sh 79 # -------------------- 80 # 81 # This generated script will remove the default routing information and then 82 # add the 10 Gb (bond1) gateway as the default route. It should be executed 83 # after the service network restart has completed. 84 # 85 # add10gIPToHosts.sh 86 # ------------------ 87 # 88 # Optional script that when executed will add entries to the hosts file in the 89 # format : 90 #91 # 10.131.12.61 el2bcn01-10g 92 # The script is simply a set of echo commands to the /etc/hosts file and it is 93 # left to the user to decide if this should be executed via dcli. 94 # 95 # Version History 96 # 1.0.0 - Initial Realease. 97 # 1.0.1 - Enable generation of add10gIPToHosts.sh script. 98 # 1.0.2 - Extend the generated createAllIfCfg.sh so that it will re-run the ssh 99 # trust setup removed at the end of the Exalogic Configuration Utility. 100 # This can be disabled by using the -i flag on the createAllIfCfg.sh. 101 # 102 ################################################################################ 103 104 NODELIST=nodelist 105 DCLIHOME=/opt/exalogic.tools/tools 106 VNICS1FILE= 107 VNICS2FILE= 108 VNICS3FILE= 109 VNICS4FILE= 110 IPFILE= 111 TENGBSUFFIX=-10g 112 113 bond1nm=255.255.255.0 114 bond1gw=10.131.12.1 115 116 while [ $# -gt 0 ] 117 do 118 case "$1" in 119 -d) DCLIHOME="$2"; shift;; 120 -n) NODELIST="$2"; shift;; 121 -ip) IPFILE="$2"; shift;; 122 -nm) bond1nm="$2"; shift;; 123 -gw) bond1gw="$2"; shift;; 124 -vnic1) VNICS1FILE="$2"; shift;; 125 -vnic2) VNICS2FILE="$2"; shift;; 126 -vnic3) VNICS3FILE="$2"; shift;; 127 -vnic4) VNICS4FILE="$2"; shift;; 128 -10g) TENGBSUFFIX="$2"; shift;; 129 *) echo ""; echo >&2 \ 130 "usage: $0 -d <10Gb Suffix> -n 131 echo""; exit 1;; 132 *) break;; 133 esac 134 shift 135 done 136 137 if test "$DCLIHOME" = "" 138 then 139 DCLIHOME=/opt/exalogic.tools/tools 140 fi 141 142 if test "$NODELIST" = "" 143 then 144 NODELIST=nodelist 145 fi 146 147 if test "$TENGBSUFFIX" = "" 148 then 149 TENGBSUFFIX=-10g 150 fi 151 152 vl1=${#VNICS1FILE} 153 vl2=${#VNICS2FILE} 154 vl3=${#VNICS3FILE} 155 vl4=${#VNICS4FILE} 156 157 # Check for the VNIC Files 158 if [ $vl1 -eq 0 ] || [ $vl2 -eq 0 ] 159 then 160 echo "Outputs from showvnics for Switch 1 and 2 must be provided" 161 exit 1 162 fi 163 164 if [ $vl3 -gt 0 ] && [ $vl4 -eq 0 ] 165 then 166 echo "Output from Switch 3 and 4 must be provided as a pair" 167 exit 1 168 fi 169 if [ $vl3 -eq 0 ] && [ $vl4 -gt 0 ] 170 then 171 echo "Output from Switch 3 and 4 must be provided as a pair" 172 exit 1 173 fi 174 175 # Need to validate the vnic file pairing ie there are either 2 or 4 I assume they are paired correctly 176 177 GENIFCFGSCRIPT=genIfcfgFiles.sh 178 CREATECFGFILE=createAllIfCfg.sh 179 SETDEFAULTROUTING=setDefaultRouting.sh 180 ADDTOHOSTS=add10gIPToHosts.sh 181 182 echo "#!/bin/sh" > $SETDEFAULTROUTING 183 echo "echo \"**********************\"" >> $SETDEFAULTROUTING 184 echo "echo \"** Processing \`hostname\` \"" >> $SETDEFAULTROUTING 185 echo "echo \"**********************\"" >> $SETDEFAULTROUTING 186 echo "netstat -rn" >> $SETDEFAULTROUTING 187 echo "route delete default" >> $SETDEFAULTROUTING 188 echo "route add default gw $bond1gw bond1" >> $SETDEFAULTROUTING 189 echo "netstat -rn" >> $SETDEFAULTROUTING 190 chmod +x $SETDEFAULTROUTING 191 192 193 194 # Copy Create Script to all nodes 195 DESTDIR=/tmp/GenScripts 196 197 echo "#!/bin/sh" > $CREATECFGFILE 198 chmod +x $CREATECFGFILE 199 200 # Set ssh Trust Setup 201 echo "SETUPSSHTRUST=true" >> $CREATECFGFILE 202 echo "" >> $CREATECFGFILE 203 204 echo "while [ \$# -gt 0 ]" >> $CREATECFGFILE 205 echo "do" >> $CREATECFGFILE 206 echo " case \"\$1\" in " >> $CREATECFGFILE 207 echo " -i) SETUPSSHTRUST=false;;" >> $CREATECFGFILE 208 echo " *) echo \"\"; echo >&2 \\" >> $CREATECFGFILE 209 echo " \"usage: \$0 [-i-ip -gw -nm -vnic1 ]\"" >> $CREATECFGFILE 210 echo " echo\"\"; exit 1;;" >> $CREATECFGFILE 211 echo " *) break;;" >> $CREATECFGFILE 212 echo " esac" >> $CREATECFGFILE 213 echo " shift" >> $CREATECFGFILE 214 echo "done" >> $CREATECFGFILE 215 echo "" >> $CREATECFGFILE 216 217 echo "if test \"\$SETUPSSHTRUST\" = \"true\"" >> $CREATECFGFILE 218 echo "then" >> $CREATECFGFILE 219 echo " echo \"Setting up SSH Trust\"" >> $CREATECFGFILE 220 echo " echo \"\"" >> $CREATECFGFILE 221 echo " ssh-keygen -t dsa" >> $CREATECFGFILE 222 echo " $DCLIHOME/dcli -k -g $NODELIST " >> $CREATECFGFILE 223 echo "fi" >> $CREATECFGFILE 224 echo "" >> $CREATECFGFILE 225 226 # Copy Files 227 echo "chmod +x genIfcfgFiles.sh" >> $CREATECFGFILE 228 echo "dos2unix genIfcfgFiles.sh" >> $CREATECFGFILE 229 echo "$DCLIHOME/dcli -g $NODELIST mkdir -p $DESTDIR" >> $CREATECFGFILE 230 echo "$DCLIHOME/dcli -g $NODELIST -d $DESTDIR -f genIfcfgFiles.sh" >> $CREATECFGFILE 231 echo "$DCLIHOME/dcli -g $NODELIST -d $DESTDIR -f $SETDEFAULTROUTING" >> $CREATECFGFILE 232 echo "" >> $CREATECFGFILE 233 234 # Now for each compute node we need to execute the appropriate command 235 NEWNODELIST=$NODELIST.out 236 > $NEWNODELIST 237 238 echo "#!/bin/sh" > $ADDTOHOSTS 239 chmod +x $ADDTOHOSTS 240 echo "echo \"\" >> /etc/hosts" >> $ADDTOHOSTS 241 echo "echo \"# Auto Generated Exalogic IP / Hostname for 10Gb (bond1) Connections\" >> /etc/hosts" >> $ADDTOHOSTS 242 echo "echo \"\" >> /etc/hosts" >> $ADDTOHOSTS 243 244 cnip=0 245 # We need to execute this for the Switch pairings 246 # VNICS1FILE & VNICS2FILE 247 # Optionally 248 # VNICS3FILE & VNICS4FILE 249 for ETH4FILE in $VNICS1FILE $VNICS3FILE 250 do 251 # Get ETH5FILE 252 if test "$ETH4FILE" = "" 253 then 254 break 255 else 256 if test "$ETH4FILE" = "$VNICS1FILE" 257 then 258 ETH5FILE=$VNICS2FILE 259 else 260 ETH5FILE=$VNICS4FILE 261 fi 262 fi 263 echo "Eth4 $ETH4FILE" 264 echo "Eth5 $ETH5FILE" 265 266 # Sort Input File 267 SORTEDETH4FILE=$ETH4FILE.sorted 268 sort +4 -5 $ETH4FILE > $SORTEDETH4FILE 269 270 ln=0 271 while read line 272 do 273 ln=`expr $ln + 1` 274 275 i=0 276 for el in $line 277 do 278 i=`expr $i + 1` 279 if [ $i -eq 1 ] 280 then 281 s=$el 282 break; 283 fi 284 done 285 286 if test "$s" = "---" 287 then 288 continue 289 fi 290 if test "$s" = "ID" 291 then 292 continue 293 fi 294 # It's a Compute node so lets process it 295 296 cnip=`expr $cnip + 1` 297 # Get the Compute Node and its Mac from the first VNIC File 298 # We will assume this is eth4 Mac address 299 i=0 300 for el in $line 301 do 302 i=`expr $i + 1` 303 if [ $i -eq 5 ] 304 then 305 cn=$el 306 echo "$cn" >> $NEWNODELIST 307 fi 308 if [ $i -eq 7 ] 309 then 310 ip=$el 311 fi 312 if [ $i -eq 9 ] 313 then 314 eth4mac=$el 315 fi 316 if [ $i -eq 12 ] 317 then 318 port=$el 319 fi 320 done 321 322 # Now we have the information from the first VNICs file we will process 323 # the second to get eth5 mac address. 324 #line=`grep $cn $VNICS2FILE` 325 line=`grep $cn $ETH5FILE` 326 #echo $line 327 i=0 328 for el in $line 329 do 330 i=`expr $i + 1` 331 if [ $i -eq 9 ] 332 then 333 eth5mac=$el 334 fi 335 done 336 337 # Now we need to get the Bond 1 (10 Gb) IP Address from the list in the supplied file 338 i=0 339 while read ipaddr 340 do 341 i=`expr $i + 1` 342 if [ $i -eq $cnip ] 343 then 344 bond1ip=$ipaddr 345 break 346 fi 347 done < $IPFILE 348 echo "# Call Script on $cn" >> $CREATECFGFILE 349 echo "ssh -l root $cn $DESTDIR/$GENIFCFGSCRIPT -ip $bond1ip -nm $bond1nm -gw $bond1gw -m4 $eth4mac -m5 $eth5mac -cn $cn" >> $CREATECFGFILE 350 351 cnalt= 352 cnl=${#cn} 353 if [ $cnl -gt 2 ] 354 then 355 ipos=`expr $cnl - 2` 356 cns=${cn:0:$ipos} 357 cne=${cn:$ipos} 358 cnalt=${cn/%$cne/x$cne} 359 fi 360 echo "echo \"$bond1ip $cn$TENGBSUFFIX $cnalt\" >> /etc/hosts" >> $ADDTOHOSTS 361 done < $SORTEDETH4FILE 362 #done < $VNICS1FILE 363 done 364 365 # Remove working file 366 echo "$DCLIHOME/dcli -g $NODELIST rm -r $DESTDIR" >> $CREATECFGFILE 367 368 # Report Complete 369 echo "" 370 echo "Generated $CREATECFGFILE" 371 echo "==================================" 372 echo "" 373 echo "File should be validated before executing" 374 echo "" 375 echo "The script has also create the $SETDEFAULTROUTING file that can be executed" 376 echo "to reconfigure the default routing once the $CREATECFGFILE has completed" 377 echo "its restart of the network" 378 echo "" 379 380
genIfgcfgFiles.sh
1 #!/bin/sh 2 3 ################################################################################ 4 # 5 # Exalogic EL X2-2 1.0 (Linux x86-64) Configuration Script. 6 # 7 # HEADER START 8 # 9 # THIS SCRIPT IS PROVIDED ON AN �AS IS� BASIS, WITHOUT WARRANTY OF ANY KIND, 10 # EITHER EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES THAT 11 # THE COVERED SCRIPT IS FREE OF DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR 12 # PURPOSE OR NON-INFRINGING. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE 13 # OF THE COVERED SOFTWARE IS WITH YOU. SHOULD ANY COVERED SOFTWARE PROVE 14 # DEFECTIVE IN ANY RESPECT, YOU (NOT THE INITIAL DEVELOPER OR ANY OTHER 15 # CONTRIBUTOR) ASSUME THE COST OF ANY NECESSARY SERVICING, REPAIR OR CORRECTION. 16 # NO USE OF ANY COVERED SOFTWARE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS 17 # DISCLAIMER. 18 # 19 # When distributing this Code, include this HEADER in each file. 20 # If applicable, add the following below this this HEADER, with the fields 21 # enclosed by brackets "[]" replaced with your own identifying information: 22 # Portions Copyright [yyyy] [name of copyright owner] 23 # 24 # HEADER END 25 # 26 # 27 # Copyright 2011 Andrew Hopkinson, Oracle Corporation UK Ltd. 28 # 29 ################################################################################ 30 31 32 ################################################################################ 33 # 34 # genIfcfgFiles.sh 35 # ================ 36 # 37 # Version : 1.0.0 38 # 39 # This script is used to generate ifcfg_bond1, ifcfg_eth4 and ifcfg_eth5 files 40 # in the /etc/sysconfig/network-scripts directory based on the passed parameters. 41 # 42 # The key reason for this script is to help configure the 10 Gb Networking on 43 # bond1 but also configure the redundant eth4 & eth5 network connection to our 44 # pairs of switches. 45 # 46 # Parameters 47 # 48 # -ip : The IP Address that should be assigned to the bond1 connection 49 # -nm : The Bond1 Network Mask 50 # -gw : The Gateway to be used by the 10 Gb bond1 conection. 51 # -m4 : The MAC Address to be associated with Eth4. This may have been 52 # previously configured using the genVNICsFromIBNetDiscover.sh or 53 # genVNICsFromListLinkUp.sh on the Switches. 54 # -m5 : The MAC Address to be associated with Eth5. This may have been 55 # previously configured using the genVNICsFromIBNetDiscover.sh or 56 # genVNICsFromListLinkUp.sh on the Switches. 57 # 58 # Processing 59 # 60 # On execution the script will first edit the /etc/sysconfig/network file to add 61 # the specified 10 Gb Gateway. Once this has been done it will create the three 62 # ifcfg- files in /etc/sysconfig/network-scripts. 63 # 64 # Having created the file the script will, optionally, then edit the /etc/sysctl.conf 65 # file to set net.ipv4.conf.default.rp_filter = 2 which will provide a loose 66 # filtering on the network and hence allow the standard management connection to be 67 # discovered and connected to. 68 # 69 # The final part of the script will restart the network service to implement the 70 # changes. On completion of this script we will need to remove the default route 71 # and add a new default that goes via the bond1 gateway. This is done in the 72 # setDefaultRouting.sh that will be generated as part of the genCNNetFiles.sh. 73 # 74 # Version History 75 # 1.0.0 - Initial Realease 76 # 77 ################################################################################ 78 79 80 # Define Variables 81 BAK_EXT=`date +"%Y%m%d-%H%M%S"` 82 BAK_PREFIX=Orig 83 84 SYSCONFROOT=/etc/sysconfig 85 IFCFGROOT=$SYSCONFROOT/network-scripts 86 87 NETWORKFILE=$SYSCONFROOT/network 88 BACKUPNETWORKFILE=$SYSCONFROOT/$BAK_PREFIX.network.$BAK_EXT 89 90 SYSCTLFILE=/etc/sysctl.conf 91 BACKUPSYSCTLFILE=/etc/$BAK_PREFIX.sysctl.conf.$BAK_EXT 92 93 BOND1IP= 94 BOND1NM= 95 BOND1GW= 96 ETH4MAC= 97 ETH5MAC= 98 99 COMPUTENODE= 100 101 # Setting the following variable to true will cause the /etc/sysctl.conf file 102 # to be edited so that the net.ipv4.conf.default.rp_filter = 2 103 # This is not recommended for production. 104 CHANGESYSCTL=false 105 106 # Read Parameters 107 while [ $# -gt 0 ] 108 do 109 case "$1" in 110 -r) IFCFGROOT="$2"; shift;; 111 -ip) BOND1IP="$2"; shift;; 112 -nm) BOND1NM="$2"; shift;; 113 -gw) BOND1GW="$2"; shift;; 114 -m4) ETH4MAC="$2"; shift;; 115 -m5) ETH5MAC="$2"; shift;; 116 -cn) COMPUTENODE="$2"; shift;; 117 *) echo ""; echo >&2 \ 118 "usage: $0 [-r] -ip 119 echo""; exit 1;; 120 *) break;; 121 esac 122 shift 123 done 124 125 echo "" 126 echo "***** Starting Processessing on Compute Node $COMPUTENODE *****" 127 echo "" 128 129 # Set File Names 130 131 IFCFG_BOND1=$IFCFGROOT/ifcfg-bond1 132 IFCFG_ETH4=$IFCFGROOT/ifcfg-eth4 133 IFCFG_ETH5=$IFCFGROOT/ifcfg-eth5 134 135 BAK_IFCFG_BOND1=$IFCFGROOT/$BAK_PREFIX.ifcfg-bond1.$BAK_EXT 136 BAK_IFCFG_ETH4=$IFCFGROOT/$BAK_PREFIX.ifcfg-eth4.$BAK_EXT 137 BAK_IFCFG_ETH5=$IFCFGROOT/$BAK_PREFIX.ifcfg-eth5.$BAK_EXT 138 139 # Backup any existing files 140 if [ -f $IFCFG_BOND1 ] 141 then 142 mv $IFCFG_BOND1 $BAK_IFCFG_BOND1 143 fi 144 if [ -f $IFCFG_ETH4 ] 145 then 146 mv $IFCFG_ETH4 $BAK_IFCFG_ETH4 147 fi 148 if [ -f $IFCFG_ETH5 ] 149 then 150 mv $IFCFG_ETH5 $BAK_IFCFG_ETH5 151 fi 152 153 # Generate Bond1 (10 Gb Ethernet File) 154 155 > $IFCFG_BOND1 156 echo "DEVICE=bond1" >> $IFCFG_BOND1 157 echo "IPADDR=$BOND1IP" >> $IFCFG_BOND1 158 echo "NETMASK=$BOND1NM" >> $IFCFG_BOND1 159 echo "BOOTPROTO=none" >> $IFCFG_BOND1 160 echo "USERCTL=no" >> $IFCFG_BOND1 161 echo "TYPE=Ethernet" >> $IFCFG_BOND1 162 echo "ONBOOT=yes" >> $IFCFG_BOND1 163 echo "IPV6INIT=no" >> $IFCFG_BOND1 164 echo "BONDING_OPTS=\"mode=active-backup miimon=100 downdelay=5000 updelay=5000\"" >> $IFCFG_BOND1 165 # Do I need the Gateway or just set it in /etc/sysconfig/network as per EIS Checklist 166 echo "GATEWAY=$BOND1GW" >> $IFCFG_BOND1 167 168 # Process Network file 169 # Backup 170 mv $NETWORKFILE $BACKUPNETWORKFILE 171 172 while read line 173 do 174 len=${#line} 175 if [ $len -eq 0 ] 176 then 177 continue 178 fi 179 f7=`expr substr $line 1 7` 180 if test "$f7" != "GATEWAY" 181 then 182 echo $line >> $NETWORKFILE 183 fi 184 done < $BACKUPNETWORKFILE 185 echo "GATEWAY=$BOND1GW" >> $NETWORKFILE 186 echo "GATEWAYDEV=bond1" >> $NETWORKFILE 187 188 echo "Edited $NETWORKFILE file" 189 190 # Generate ETH4 191 192 > $IFCFG_ETH4 193 echo "DEVICE=eth4" >> $IFCFG_ETH4 194 echo "BOOTPROTO=none" >> $IFCFG_ETH4 195 echo "ONBOOT=yes" >> $IFCFG_ETH4 196 echo "HWADDR=$ETH4MAC" >> $IFCFG_ETH4 197 echo "MASTER=bond1" >> $IFCFG_ETH4 198 echo "SLAVE=yes" >> $IFCFG_ETH4 199 200 # Generate ETH5 201 202 > $IFCFG_ETH5 203 echo "DEVICE=eth5" >> $IFCFG_ETH5 204 echo "BOOTPROTO=none" >> $IFCFG_ETH5 205 echo "ONBOOT=yes" >> $IFCFG_ETH5 206 echo "HWADDR=$ETH5MAC" >> $IFCFG_ETH5 207 echo "MASTER=bond1" >> $IFCFG_ETH5 208 echo "SLAVE=yes" >> $IFCFG_ETH5 209 210 211 # Complete Message 212 213 echo "Generated New ifcfg files" 214 echo " $IFCFG_BOND1" 215 echo " $IFCFG_ETH4" 216 echo " $IFCFG_ETH5" 217 218 if test "$CHANGESYSCTL" = "true" 219 then 220 # Change the /etc/sysctl.conf so that the default filtering is 2 221 # This will allow the normal & 10 Gb network access 222 223 # Backup 224 mv $SYSCTLFILE $BACKUPSYSCTLFILE 225 226 while read line 227 do 228 len=${#line} 229 if [ $len -eq 0 ] 230 then 231 echo "" >> $SYSCTLFILE 232 fi 233 234 i=0 235 for el in $line 236 do 237 i=`expr $i + 1` 238 if [ $i -eq 1 ] 239 then 240 if test "$el" = "net.ipv4.conf.default.rp_filter" 241 then 242 echo "net.ipv4.conf.default.rp_filter = 2" >> $SYSCTLFILE 243 else 244 echo $line >> $SYSCTLFILE 245 fi 246 break 247 fi 248 done 249 done < $BACKUPSYSCTLFILE 250 251 echo "Editted $SYSCTLFILE file" 252 fi 253 254 # Restarting Network 255 256 service network restart 257 258 echo "" 259 echo "***** Finished Processessing on Compute Node $COMPUTENODE *****" 260 echo "" 261 262-nm -gw -m4 -m5 "
vIPs
Virtual IPs are useful when multiple, virtual, IPs for a single compute node and we have used this when running multiple WLS domains on the same Compute Node. Using vIPs allows you the create all the domains with the same port number but different IP address. The net result is that we can simplify HA / Failover and load balancing.The simple script below will generate a specified number of, sequential, vIPs with a given Netmask for a given bond.
genVNICsFromLinkListUp.sh
1 #!/bin/sh 2 3 ################################################################################ 4 # 5 # Exalogic EL X2-2 1.0 (Linux x86-64) Configuration Script. 6 # 7 # HEADER START 8 # 9 # THIS SCRIPT IS PROVIDED ON AN �AS IS� BASIS, WITHOUT WARRANTY OF ANY KIND, 10 # EITHER EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES THAT 11 # THE COVERED SCRIPT IS FREE OF DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR 12 # PURPOSE OR NON-INFRINGING. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE 13 # OF THE COVERED SOFTWARE IS WITH YOU. SHOULD ANY COVERED SOFTWARE PROVE 14 # DEFECTIVE IN ANY RESPECT, YOU (NOT THE INITIAL DEVELOPER OR ANY OTHER 15 # CONTRIBUTOR) ASSUME THE COST OF ANY NECESSARY SERVICING, REPAIR OR CORRECTION. 16 # NO USE OF ANY COVERED SOFTWARE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS 17 # DISCLAIMER. 18 # 19 # When distributing this Code, include this HEADER in each file. 20 # If applicable, add the following below this this HEADER, with the fields 21 # enclosed by brackets "[]" replaced with your own identifying information: 22 # Portions Copyright [yyyy] [name of copyright owner] 23 # 24 # HEADER END 25 # 26 # 27 # Copyright 2011 Andrew Hopkinson, Oracle Corporation UK Ltd. 28 # 29 ################################################################################ 30 31 32 ################################################################################ 33 # 34 # createVLANs.sh 35 # ============== 36 # 37 # Version : 1.0.0 38 # 39 # This scripts simply creates a number of VLANs for a specified bond. The script 40 # must be provided with all the parameters and will then simply loop starting at 41 # the first IP Address and incrementing by 1 until it has processed the specified 42 # number. 43 # 44 # Parameters 45 # 46 # -ip : Starting IP Address in the format 192.168.12.1. 47 # -nm : Netmask (i.e. 255.255.255.0) 48 # -c : Number of VLANs to create 49 # -b : Bond on which the VLANs should be created (bond0 or bond1) 50 # 51 # Version History 52 # 1.0.0 - Initial Realease 53 # 54 ################################################################################ 55 56 57 # functions 58 usage() 59 { 60 echo "" 61 echo >&2 "usage: $0 [-ip Starting IP] [-nm Netmask] [-c Number of IPs require] [-b Bond]" 62 echo"" 63 } 64 65 SBIN=/sbin 66 67 while [ $# -gt 0 ] 68 do 69 case "$1" in 70 -ip) STARTIP="$2"; shift;; 71 -nm) MASK="$2"; shift;; 72 -c) COUNT="$2"; shift;; 73 -b) BOND="$2"; shift;; 74 -sbin) SBIN="$2"; shift;; 75 *) usage; exit 1;; 76 *) break;; 77 esac 78 shift 79 done 80 81 if test "$STARTIP" = "" 82 then 83 echo "Starting IP Address must be specified" 84 usage 85 exit 1; 86 fi 87 if test "$MASK" = "" 88 then 89 echo "Netmask must be specified" 90 usage 91 exit 1; 92 fi 93 if test "$COUNT" = "" 94 then 95 echo "Number of IP Addresses must be specified" 96 usage 97 exit 1; 98 fi 99 if test "$BOND" = "" 100 then 101 echo "Bond must be specified" 102 usage 103 exit 1; 104 fi 105 106 # Strip out the constituent parts of the IP Address 107 ip=$STARTIP 108 for i in 1 2 3 109 do 110 pos=`expr index $ip .` 111 if [ $i -eq 1 ] 112 then 113 ip1=${ip:0:$pos} 114 fi 115 if [ $i -eq 2 ] 116 then 117 ip2=${ip:0:$pos} 118 fi 119 if [ $i -eq 3 ] 120 then 121 ip3=${ip:0:$pos} 122 fi 123 if [ $i -eq 4 ] 124 then 125 ip4=${ip:0:$pos} 126 fi 127 ip=${ip:$pos} 128 ip4=$ip 129 done 130 131 # Loop through and create VLANS 132 for (( i=1; i<=$COUNT; i++ )) 133 do 134 ip=$ip1$ip2$ip3$ip4 135 $SBIN/ifconfig $BOND:$i $ip netmask $MASK up 136 $SBIN/arping -q -U -c 3 -I $BOND $ip 137 ip4=`expr $ip4 + 1` 138 done 139 140 ip4=`expr $ip4 - 1` 141 ENDIP=$ip1$ip2$ip3$ip4 142 143 echo "Creating a ifcfg file to make permenant" 144 echo "" 145 146 BAK_EXT=`date +"%Y%m%d-%H%M%S"` 147 IFCFG_FILE="/etc/sysconfig/network-scripts/ifcfg-$BOND" 148 VIP_RANGE_FILE="$IFCFG_FILE-range0" 149 BACKUP_VIP_RANGE_FILE=Orig.$VIP_RANGE_FILE.$BAK_EXT 150 mv $VIP_RANGE_FILE $BACKUP_VIP_RANGE_FILE 151 152 while read line 153 do 154 f6=${line:0:6} 155 if test "$f6" = "IPADDR" 156 then 157 echo $line 158 else 159 echo $line >> $VIP_RANGE_FILE 160 fi 161 done < $IFCFG_FILE 162 163 echo "IPADDR_START=$STARTIP" >> $VIP_RANGE_FILE 164 echo "IPADDR_END=$ENDIP" >> $VIP_RANGE_FILE 165 echo "CLONENUM_START=1" >> $VIP_RANGE_FILE 166 167 echo "" 168 echo "Created VLANs $STARTIP to $ENDIP on $BOND" 169 echo "" 170 171
vLANs
Virtual LANs are used to combine multiple domains and hence IP subnets onto a single physical port. The be fully configured both ends of the connected cable must be capable and configured for vLANs and hence I will assume this is the case. We use vLANs to enforce security isolation, division of workload and splitting traing across multiple domains / subnets.A simple scenario (I will document more in later articles) would be to partition the Exalogic so that we can run Production, Test and Development on the same box but keep their 10Gb Ethernet connection isolated.
To create a vLAN (no script for this one) we simply issue the createvlan command on the switch as follows.
createvlanWhere:-vlan -pkey default
- ETH Port : Ethernet Port on the switch e.g. 0A-ETH-3, 1A-ETH-1, etc
Unique number between 2 and 4094
At this point to enforce the network issolation we will associated the newly created vLANs to specific vNICs and hence Compute Nodes. At present the scripts for creating vNICs do not do this but it can be easily achived by modifying the generated Create-
No comments:
Post a Comment