Following the success of the CreateAssets shell script and it's subsequent merging into the SimpleExaCli script I decided to extend the features and leverage the full functionality of the 2.0.6.x (Echo) IaaS. As part of the revisit of the code a rewrite in python was performed and the primary reason for this was it allowed for simple coding to read and access the new JSON format definition file. The additional flexibility that the python language allows provided a framework within which we can build a simple assembly builder.
Because I have chosen to use Python 3 and the pexpect libraries this version of the CreateAssets functionality will only be released as a vServer Template, that is an extension of the Echo template described here, and can be downloaded from the link below.
The rest of this blog entry will describe the new calling mechanism and focus on the new structure within the JSON files. The scripts and files described can all be found within the examples directory in the template.
The example scripts provided have been leveraged from other sources / documentation and are subject to change between releases of the template as the example functionality is modified and extended.
Because I have chosen to use Python 3 and the pexpect libraries this version of the CreateAssets functionality will only be released as a vServer Template, that is an extension of the Echo template described here, and can be downloaded from the link below.
The rest of this blog entry will describe the new calling mechanism and focus on the new structure within the JSON files. The scripts and files described can all be found within the examples directory in the template.
CreateAssembly.sh & CreateAssembly.py
CreateAssembly.py
Main python script that will process the passed JSON file and apply to the EMOC specifiedUsage:
python3 CreateAssembly.py -u <User> -p <Password File> -f <Assembly File> --url <EMOC URL>
Mandatory Parameters
-u, --user : Username to be used to access the EMOC via the IaaS Cli.
-p, --passwordFile : Name of the file containing the password associated with the user, this file will contain just the password.
-f, --assemblyFile : JSON file containing the asset and assembly information.
--url : URL to access EMOC.
Optional Parameters
-h, --help : Print this help.
-d, --debug : Prints verbose debug output during execution.
--history : Prints script version history.
CreateAssembly.sh
Simple wrapper around the CreateAssembly.py.PYTHON_SCRIPT_DIR=`dirname $0` python3 $PYTHON_SCRIPT_DIR/CreateAssembly.py "$@"
Assembly JSON File
The JSON file defines the Assets and Assemblies to be built byt the python script. In the following Example JSON file we can see that it is essentially split into two main sections:- "assets" : This contains an array of all asset types to be created (networks, templates, volumes and distribution groups).
- "assembilies" : Contains a list of vServers and configurations to be created.
{
"date": "04/11/2013",
"version": "0",
"assets": [
{
"account": "Account Name",
"networks": [
{
"name": "Network Name",
"size": "Number of IPs"
}
],
"templates": [
{
"name": "Template Name",
"url": "Location "
}
],
"volumes": [
{
"name": "Volume Name",
"size": "Size of Template"
}
],
"distributiongroups": [
{
"name": "Distribution Group Name",
"size": "Number of Nodes"
}
]
}
],
"assemblies": [
{
"name": "Assembly Name",
"account": "Account Name",
"transaction": "Rollback",
"vservers": [
{
"name": "vServer Name",
"type": "vServer Type Name",
"template": "Template Name",
"distributiongroup": "Distribution Group Name",
"ha": "true",
"hostname": "Hostname if different to default",
"networks": [
{
"name": "Network Name",
"ip": "Static IP addres or 'auto'",
"hostname": "Hostname if network specific one is required"
}
],
"volumes": [
{
"name": "Volume Name"
}
],
"description": "Description"
}
],
"configurations": [
{
"vserver": "vServer Name",
"scripts": [
{
"name": "Script to be execute",
"local": "Indicated if this must be copied to the vServer 'true'",
"parameters": [
{
"name": "ip",
"type": "reference",
"vserver": "Referenced vServer Name",
"network": "Network associated with the referenced vServer",
"attribute": "ip"
},
{
"name": "hostname",
"type": "static",
"value": "otdSecondaryServer"
}
]
}
]
}
]
}
]
}
"assets":
This section contain information about the EMOC assets that will be created or upload. Because the IaaS limits us to "Cloud User" level actions we are restricted on the asset we can create to:- Networks (Private VNets).
- Templates (Upload).
- Volumes.
- Distribution Groups.
"networks":
The networks is an array object within the file and allows for multiple private vNets to be defined. We can see from the JSON snippet below that for each network we can specify:- "name": This defines the display name of the network.
- "size": Specifies the number of IP addresses to allocate to this network.
"networks": [
{
"name": "vNetOTD",
"size": "200"
},
{
"name": "vNetWLS",
"size": "200"
}
],
"templates":
This is a JSON array object contains a list of template object each of which will cause the python to import the specified .tgz into the account. The snippet below shows we can specify:- "name": Display name of the template.
- "url": URL to the .tgz to be imported, generally located on the ZFS.
"templates": [
{
"name": "OTDTestTemplate",
"url": "http://172.17.0.5/shares/export/common/images/vServerTemplates/el_template_OTDTemplateServer.tgz"
},
{
"name": "WLSTestTemplate",
"url": "http://172.17.0.5/shares/export/common/images/vServerTemplates/el_template_WLSTemplateServer.tgz"
}
],
"volumes":
The volumes array contains a list of volumes to be created and there associated size and the snippet below shows we can specify:- "name": Display name of the volume
- "size": Size of volume in Gb
"volumes": [
{
"name": "PYTestVol1",
"size": "20"
},
{
"name": "PYTestVol2",
"size": "40"
}
],
"distributiongroups":
Within the distributiongroups array we can specify multiple Distribution Groups. Because we are using the 2.0.6.x (Echo) IaaS we can specify size of these groups through the IaaS. The snippet of JSON below shows what can be specified:- "name": Display name of the Distribution Group.
- "size": Number of nodes in the Distribution Group.
"distributiongroups": [
{
"name": "dgOTD",
"size": "8"
},
{
"name": "dgWLS",
"size": "8"
}
]
"assemblies":
The assemblies section of the JSON file describes the vServers that will be created as part of the Assembly. Again this is defined as a JSON array so we can process multiple accounts. Each assembly object within the array contains the following information:- "name": Descriptive name of this assembly.
- "account": Name of the account with which the vServers will be created.
- "transaction": Currently not used but is in place for future enhancement.
- "vservers": Array of vServer information and will be discussed later in the blog.
- "configurations": Array of configuration scripts to execute along with there parameters and will be discussed later in the blog.
"vservers":
The vservers array contains one or many vserver JSON objects that are similar to the snippet below. It can be seen from the snippet that each vServer can contain the following information:- "name": (Mandatory) vServer display name and by default hostname.
- "type": (Mandatory) vServer Type
- "template": (Mandatory) Name of the template to be used to create the vServer
- "ha": (Optional) HA status of the vServer and will default to true.
- "hostname": (Optional) Hostname to be used for the vServer it will default to name of the vServer.
- "networks": Array of network the vServer needs to have defined.
-
- "name": Name of the network to which the vServer needs to connect.
- "ip": Specify either the static IP to be used or if automatic allocation is to be used then "auto" is specified.
- "hostname": (Optional) hostname to be used specifically for this network.
- "volumes": (Optional) Volumes to be assigned to this vServer
-
- "name": Name of the volume to be attached
- "description": Freeform description for the vServer.
{
"name": "otdAdminServer",
"type": "SMALL",
"template": "OTDTestTemplate",
"distributiongroup": "dgOTD",
"ha": "true",
"hostname": "otdAdminServer",
"networks": [
{
"name": "vNetOTD",
"ip": "auto",
"hostname": "otd-admin"
},
{
"name": "vNetWLS"
},
{
"name": "IPoIB-vserver-shared-storage",
"ip": "172.17.0.42"
},
{
"name": "scae09-eoib1",
"ip": "10.128.36.202"
}
],
"description": "OTD Admin vServer"
},
"configurations":
The configuration section is used to define a number of post creation configuration scripts that will be executed once the vServers are in a RUNNING state. For each server, specified, a number of scripts can be defined for execution (processed in the order specified within the JSON). Each configuration object specifies:- "vserver": Name of vServer against which the script will be run.
- "scripts": Array of scripts to be run on the vServer. (See below)
{
"vserver": "otdAdminServer",
"scripts": [
{
"name": "/root/examples/scripts/addIpToHosts.sh",
"local": "true",
"parameters": [
{
"name": "ip",
"type": "reference",
"vserver": "wlsManagedServer1",
"network": "vNetWLS",
"attribute": "ip"
},
{
"name": "hostname",
"type": "static",
"value": "wlsManagedServer1"
}
]
},
{
"name": "/root/examples/scripts/addIpToHosts.sh",
"local": "true",
"parameters": [
{
"name": "ip",
"type": "reference",
"vserver": "wlsManagedServer2",
"network": "vNetWLS",
"attribute": "ip"
},
{
"name": "hostname",
"type": "static",
"value": "wlsManagedServer2"
}
]
},
"scripts":
The scripts object defines the information required to allow execution of the script on the vServer. The information within this object specifies the following:- "name": This defines the name of the script it can be either relative or full path to the script. In addition this can refer to either a location on the local machine or on the vServer.
- "local": A flag to indicate if this script is on the local machine (true) or remote machine (false). If this is local to the script then it will be copied to the vServer before execution.
- "parameters": An array of parameters to be passed to the script the python will take the names of the parameter and pass it in the command line with a "--" prefix.
-
- "name": Name of the parameter
- "type": Defines if this is a static or reference parameter. If this is a reference then the name of the referenced vServer must be supplied along with the network and attribute. In this first release we are limited to retrieving the IP address for a given network of the specified vServer.
- "vserver": Name of vServer to be referenced.
- "network": Name of the network attached to the vServer
- "attribute": name of the attribute to be retrieved at present this is restricted to "ip"
What's in the Template
The template contains a pre-installed environment consisting of python3 and the associated CreateAssembly.sh and CreateAssembly.py. In addition to these an examples directory has been packaged that contains a simple ExalogicAssembly.json that builds 3 OTD vServers and 3 WLS vServer installing the products and creating nodes. The locations of these shares are defined within the JSON file.The example scripts provided have been leveraged from other sources / documentation and are subject to change between releases of the template as the example functionality is modified and extended.
[root@PythonEchoUtilityServer ~]# tree examples/
examples/
|-- json
| |-- ExalogicAssembly.json
| `-- ExalogicAssemblyTemplate.json
`-- scripts
|-- ModifyLVMImg.sh
|-- MountSystemImg.sh
|-- SimpleExaCli.sh
|-- addIpToHosts.sh
|-- configureWLSDomainAutoStart.sh
|-- copyFilesToPythonVServer.sh
|-- createOTDAdminNode.sh
|-- createOTDAdminServer.sh
|-- createOTDConfiguration.sh
|-- createOTDFailoverGroup.sh
|-- createWLSDomain.sh
|-- el-auto.tgz
|-- installOTD.sh
|-- installWLS.sh
|-- mountOTDShares.sh
|-- mountWLSShares.sh
|-- setupCustomIPoIB.sh
`-- silent.xml
2 directories, 19 files
No comments:
Post a Comment