Jul
    13

    First HCI Vendor in the Garnter Magic Quadrant with Native Local Key Manager, New in AOS 5.8

    With the release of AOS 5.8 Nutanix brings to market the first native key manager for a HCI that goes beyond using local server management tools. To help reduce cost and complexity Nutanix added a native Local Key Manager(LKM) for all three node clusters and above. The local Key manager runs as a service distributed among all of the nodes. It is easily activated from within prism element, so all customers can enable encryption without yet another silo to manage. Customers looking to simplify their infrastructure operations can now have one click infrastructure for their key manager as well.

    Usually External Key Managers (EKM) need to be purchased separately for software and hardware costs. Since the Nutanix LKM is running natively within the controller virtual machine(CVM) it’s highly available and there is not a variable add-on pricing based on the number of nodes. Every time you add a node you know the final cost. There is also peace of mind when you go to upgrade your cluster that the key management services are also going to be upgraded. By both having the infrastructure and management services upgraded in lockstep you’re assured of your security posture and availability by staying in line with the support matrix.

    The native LKM service uses the FIPS 140 Crypto module to keep all of the data encryption keys safe. Data is encrypted using a data encryption key(DEK). There is DEK used for every storage container. The DEK is typically then encrypted by a Key Encryption Key (KEK) that is sent to a EKM. Know that Nutanix supports it’s own native LKM, we take the KEK and wraps a 256 bit encryption key called the Machine Encryption Key (MEK). The MEK is distributed amongst all of the CVMs in the cluster using a splitting algorithm. No separate virtual machines are needed to support the native LKM.

    Since the MEK is shared, each node can read what others have written. For the keys to reconstruct a majority of the nodes need to be present. We use K = Ceiling(N/2) to determine the majority of nodes. So in a 11 node cluster we would need six nodes to un-encrypt the data where N is 11.

    EKM and LKM work flows

    Nutanix also provides an easy way to back up your Data Encryption Keys (DEK) from Prism. There will be DEK for each storage container. If a new storage container is created an alert will be generated encouraging administrators to take a backup. The backup is password protected and should be securely stored. With the backup in hand, if catastrophic event happened in your data centre you could replicate the data back and reimport the backup keys to get your environment up and running.

    Data

    Nutanix Backup for DEK

    Is Nutanix Local Key Manager is another step towards enabling security for everyone. Stay safe and if you have questions please drop them in the comments.

    Jun
    25

    RHEL 7 STIG Implementation in Nutanix CVM #Security

    Nutanix leverages SaltStack and SCMA to self-heal any deviation from the security baseline configuration of the operating system and hypervisor to remain in compliance. If any component is found as non-compliant, then the component is set back to the supported security settings without any intervention. To achieve this objective, Nutanix has implemented the Controller VM to support STIG compliance with the RHEL 7 STIG as published by DISA. Acropolis Operating System (A)S) 5.1 was the last version that we published our own STIGs for AOS. 5.5.3+ and 5.6+ are aligned to the RHEL 7 STIG.

    The Nutanix platform and all products leverage the Security Configuration Management Automation (SCMA) framework to ensure that services are constantly inspected for variance to the security policy. Nutanix has implemented security configuration management automation (SCMA) to check multiple security entities for both Nutanix storage and AHV. Nutanix automatically reports log inconsistencies and reverts them to the baseline. With SCMA, you can schedule the STIG to run hourly, daily, weekly, or monthly. STIG has the lowest system priority within the virtual storage controller, ensuring that security checks do not interfere with platform performance.

    The STIG rules are capable of securing the boot loader, packages, file system, booting and service control, file ownership, authentication, kernel, and logging.

    Example: STIG rules for Authentication
    Prohibit direct root login, lock system accounts other than root, enforce several password maintenance details, cautiously configure SSH, enable screen-locking, configure user shell defaults, and display warning banners.

    Nutanix has taken this ownership on, not to put the burden on our customers to take the responsibility for securing our own product. You’re only as secure as the last time you checked.

    Jun
    24

    Rubrik Extending Deeper into the Enterprise Data Center.

    Rubrik is committed to providing the same powerful UX and simpler interface for traditional enterprise workloads. Rubrik nows protect AIX and Oracle Solaris,
    helping backup admins skip a generation and migrate that data to their storage archive of choice. I particularly like this because how Rubrik can be used to backup both AHV on x86 and on Power. Rubrik also announced support for AHV VSS snapshots.

    For AIX support crafted connector with low resource and operational overhead. The connector supports granular folder/file level backups via filesets that integrate with SLA policies. Filesets are a logical construct based on Includes and Excludes. These are often around a direct path to a file or folder but can use wildcards for greater flexibility around what is included or excluded.

    For example, you might backup /home on all servers on a daily basis, excluding .mp3 files. At the same time, you might backup other folders on the same servers on a weekly or hourly basis. Filesets can be customized to meet specific needs. While we recommend keeping them simple, there is a high degree of customization possible, especially via calling Pre/Post Scripts. Pre/Post scripts can also be configured to provide application-consistent backups.

    For AIX, we’ve Rubrik’s initial support on the versions (7.1 and 7.2) and filesystems (JFS and JFS2).

    AHV VSS Snapshots

    While Rubrik has provided support for Nutanix AHV VMs since version 4.0, adaptive consistency for AHV VMs
    is now provided in the following order of precedence:

    1. Rubrik Backup Service (RBS) based VSS snapshot
    2. Native AHV application-consistent snapshot
    3. Crash-consistent snapshot

    With AHV environments, Rubrik leverages Nutanix’s REST 3.0 API to drive an initial full backup followed by incremental-forever backups in the most efficient manner possible. Rubrik CDM interacts with Nutanix Changed Region Tracking (CRT) to query only the changed metadata regions between any two snapshots of a VM or virtual disk. The efficiencies gained through this approach are essential to taking incremental backups, and provide the added benefit of saving on read operations for full backups. Utilization of parallel streams to rapidly ingest data magnifies these efficiencies, minimizing impacts to production workloads to near-zero levels. Supporting this level of performance at scale, Rubrik’s distributed architecture enables linear scaling of
    throughput processes. Though straightforward in its architecture, the Rubrik integration with Nutanix delivers granular RPOs and application-consistent backups while automating the protection and restore workflows. With no hypervisor stun and both companies parallel streaming approach rapidly ingests the snapshot disks’ data into Rubrik’s Atlas filesystem.

    Additionally, using VSS snapshots with Rubrik also provides app-consistent log truncation for Exchange – it automatically check to see if logs are backed up, and if done, truncate the logs up to the nearest point when the backup commenced. Rubrik doesn’t require specific settings to be turned on to allow app-consistent log truncation. Log truncation prevents the VM from running out of disk space and eventually crashing.

    Jun
    22

    Nutanix Acropolis Files Services (SMB/NFS) adds Health Page for Advanced troubleshooting in 3.0.1

    With the release of 3.0.1 Nutanix AFS has added a health page for advanced troubleshooting. While it’s probably not the sexiest thing in the world it does put it on par with similar pages that we have today for Stargate like the 2009 page and Curator on 2010. Since this is more for support I think it’s okay that it’s not the same as Prism. It’s always a fine line in the UI on what to add and what to leave out. You can find more information about the existing troubleshooting pages on the Nutanix Bible.

    The AFS health page is a FSVM-level webpage showing statistics and health related information about major AFS components. It runs at the default AFS port: 7502. It can be accessed using links CLI browser or web browser at http://:7502. The Internal IP would be best choice to use and I would firewall off the external port.

    Below are some examples of what you will find when you go to the AFS Health Page.

    Jun
    20

    Choice With Nutanix, Agility with Docker EE, Security with Both and all thanks to the French!

    If I took the great detective skills from the CNN and Fox News it would be easy to come to the conclusion that containers were invented by the French. I can come to this conclusion because coworker and fellow booth babe in crime Christophe Jauffret is French and so is Solomon Hykes. While I didn’t see Solomon at DockerCon 2018, Christophe built a great little demo using Nutanix Calm with Docker EE which fit into the conferences maintain theme, choice, agility, security.
    While I’m not totally delusional I know Docker wasn’t talking directly about Nutanix but two companies do share the same thoughts on the theme they selected.

    Choice: Docker EE is not tied to a single OS or VM model. Docker also supports the leading cloud hosted Kubernetes services. Nutanix supports multiple hypervisor’s four on Prem which shines in the infrastructure as a service arena but can also use Nutanix Calm to deploy to GCP or AWS using a common blueprint.

    Agility: Docker EE has proven ROI and infrastructure savings that was proven when McKesson got on stage. Docker EE accelerates on-boarding with uniform operating model across clouds and helps with automation of application life-cycle policies. Mechanics allows you to focus on the application while we take care of the infrastructure stack. With the Nutanix calm you can use blueprints from the marketplace or share them on the community.

    Security: Nutanix takes security very seriously. Nutanix has automated Security Technical Implementation Guides that keep your system in a hardened state. Not only can these STIGS automatically run hourly, daily, weekly, they are tested across the whole stack so nothing breaks and takes Nutanix customers out of the guessing game and dealing with pages of Excel spreadsheets with changes to make. Docker with its trusted registry gives image scanning and integrated security policies across clouds. Together Nutanix and docker can secure your entire build pipeline.

    With all that being said let’s get to the cool part, the demo.

    Christophe has built a blueprint in Calm to deploy one master node and two worker nodes. This could be easily changed to three master nodes and he also shows auto scaling the worker nodes. When you watch the demo you are able to see all of the steps that are happening, you can check the logs for any errors that happened, you can add variables for versions, protect passwords and add steps for full life-cycle management.

    In the demo he is also using Nutanix AFS which provides SMB and NFS storage. He has set up a storage class so that persistent volume claims can be used. Since AFS can have multiple NAS heads, you can have great performance for building environments when using our Shared export type. The Shared export type allows for all top level directories in the root of the export to land on different NAS heads. While it is not shown in the demo Nutanix can provide block support as well and has a AHV turbo for great performance.

    Nutanix Calm Blueprint

    The biggest announcement at DockerCon was the ability for Docker EE to support federated management across hybrid and multi-cloud infrastructure. Nutanix can help deploy EE on Prem or to GCP and or AWS in the cloud. If you want to run containers on AKS or EK S you can do that to. Wherever you need your application to be ran with the proper security controls in place, it’s possible without being locked in.

    If you want to get started today you can download Nutanix Community Edition and give it a shot today, Nutanix Calm licenses are included, security is on by default and AHV it is integrated for seamless experience.

    Happy Computing!

    Jun
    19

    Nutanix NAS AFS 3.0.1 Released with Support for Active/Active with Peer Software

    If you’re the type of person that needs to wait for the dot release after the GA, this is for you! At GA of AFS 3.0, Nutanix released File auditing API’s for AFS to be used by 3rd party vendors. Peer Software is using these API’s to provide an active/active solution for your SMB needs. With these API’s, Peer and Nutanix can offer endless solutions for your file severing needs. Migration, building upon an existing solution or a straight up Nutanix to Nutanix solution is possible with the lowest recovery time for your customers.

    Effective data protection for remote offices
    WAN / Latency Tolerance
    Support for heterogeneous vendor/platforms

    PeerGFS is a software-based data management solution that works to create a unified globally accessible file system that provides users with fast local access to their data while simultaneously enabling continuous protection and high availability.

    Note:
    With this release AFS supports the following antivirus software:

    Kaspersky Security 10 for Windows Server
    Sophos Endpoint Security and Control 9, 10
    Symantec Protection Engine 7.9.0
    McAfee VirusScan Enterprise for Storage 1.2.0 and 1.3.0

    Jun
    06

    Kubernetes Tips #1: security authn/authz for etcd

    According to the CIS `–kubelet-client-certificate` and `–kubelet-client-key` arguments: “By default, certificate-based kublet authentication is not set.” The requests from the apiserver are treated anonymously.

    Without those, the API server makes anonymous requests to the kubelet to do things like pod exec and logs requests. If that works, that means the kubelet has not enabled any authn/authz protections and anyone with network visibility to it can make those same exec/logs requests against it.

    The above flags you want those on the API server, you also really want `–anonymous-auth=false` and `–authorization-mode=Webhook` on the Kubelet to enable authn/authz.

    Thanks to Jordan Liggitt and Brad Geesaman

    May
    23

    Nutanix Calm Blueprint for Auto Scaling Compute Resources for Cloudera

    Adding additional compute resources by first looking at seasonal trending data from Prism Central. Auto scaling is done with using Nutanix Calm.

    Save the below code as a json file.
    cdh_parcel.version:\n cdh_parcel = p\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" downloaded\"\n\n# distribute the parcel\nprint \"Starting parcel distribution. This might take a while.\"\ncmd = cdh_parcel.start_distribution()\nif cmd.success != True:\n print \"Parcel distribution failed!\"\n exit(0)\n\nprint 'wait 4 min'\n# make sure the distribution finishes\n#while cdh_parcel.stage != \"DISTRIBUTED\":\nsleep(200)\n# cdh_parcel = get_parcel(api, cdh_parcel.product, '5.13.3-1.cdh5.13.3.p0.2', cluster_name)\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" distributed\"\n\n# activate the parcel\ncmd = cdh_parcel.activate()\nif cmd.success != True:\n print \"Parcel activation failed!\"\n exit(0)\n\n# make sure the activation finishes\n#while cdh_parcel.stage != \"ACTIVATED\":\n# cdh_parcel = get_parcel(api, cdh_parcel.product, '5.13.3-1.cdh5.13.3.p0.2', cluster_name)\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" activated\"\n\n\nyarn = cluster.get_service('yarn')\nprint yarn.name\nhostsid = api.get_all_hosts()\n#range = len(hostsid)\n#print 'range: ' + range\n\nfor current in hostsid:\n if current.rackId == '/default/default':\n print 'host name to add' + current.hostname\n currentshort1 = current.hostname\n currentshort = currentshort1.split(\".\", 1)\n short = currentshort[0]\n print short\n yarn.create_role(short, 'NODEMANAGER', current.hostId)\n\ncluster.restart(restart_only_stale_services=True, redeploy_client_configuration=True)\n\nexit(0)", "script_type": "sh", "type": "", "command_line_args": "", "login_credential_local_reference": {"kind": "app_credential", "name": "go", "uuid": "f356d2b6-2fac-453c-84ad-d91bc21bb1ba"}}, "timeout_secs": "", "type": "EXEC", "variable_list": [], "uuid": "b24dd948-e65d-2c1e-52d8-c4d21d615274"}], "description": "", "name": "7da2879a_runbook_cloned_1", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "6405be68_dag", "uuid": "7ec268af-4e53-3c55-930a-740501d41cfb"}, "message_list": [], "variable_list": [], "uuid": "2363d280-af23-b97a-557f-1e6a3d156fbb"}, "type": "", "uninstall_runbook": {"task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "de4520ca_dag", "state": "ACTIVE", "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "uuid": "aabae04b-08bb-466e-76a2-1fcfc3c816cd"}], "description": "", "name": "339b443d_runbook_cloned_1", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "de4520ca_dag", "uuid": "aabae04b-08bb-466e-76a2-1fcfc3c816cd"}, "message_list": [], "variable_list": [], "uuid": "ff185ae5-bc26-095b-6840-9fc957638621"}}, "timeout_secs": "", "type": "CUSTOM_PACKAGE_INSTALL", "variable_list": [], "uuid": "47831e03-3fa0-4a8e-b26c-8244da30ab82"}], "description": "", "name": "SYS_GEN__Runbook_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "SYS_GEN__DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "078dca59-9b50-4ad8-b048-2ca148f8251e"}, "message_list": [], "variable_list": [], "uuid": "b81bf28e-424c-4daa-9beb-8ae9a62855ba"}, "message_list": [], "name": "action_install"}, {"description": "System action for creating an application", "type": "system", "uuid": "563a264b-ca8e-4279-ad98-96e4e8d77282", "state": "ACTIVE", "critical": true, "attrs": {}, "runbook": {"task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [{"kind": "app_task", "name": "SYS_GEN__install_CRb_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "d4857a2b-32f6-471d-9e05-295df9e45c87"}, {"kind": "app_task", "name": "SYS_GEN__create_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "55697484-6734-4bdd-afde-75e8c171697e"}, {"kind": "app_task", "name": "SYS_GEN__start_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "d5751cd2-f72a-4f35-9fd6-104079471e54"}], "name": "SYS_GEN__Composite_DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "attrs": {"edges": [{"from_task_reference": {"kind": "app_task", "name": "SYS_GEN__create_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "55697484-6734-4bdd-afde-75e8c171697e"}, "edge_type": "inherent", "type": "", "uuid": "873c853d-d849-46a7-9344-625a6b9c0898", "to_task_reference": {"kind": "app_task", "name": "SYS_GEN__start_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "d5751cd2-f72a-4f35-9fd6-104079471e54"}}, {"from_task_reference": {"kind": "app_task", "name": "SYS_GEN__install_CRb_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "d4857a2b-32f6-471d-9e05-295df9e45c87"}, "edge_type": "inherent", "type": "", "uuid": "d316c8f7-435c-4b1c-ad0c-7b1d02696dcd", "to_task_reference": {"kind": "app_task", "name": "SYS_GEN__create_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "55697484-6734-4bdd-afde-75e8c171697e"}}], "type": "DAG"}, "timeout_secs": "", "type": "DAG", "variable_list": [], "uuid": "dcc6c9a0-ef16-406c-8960-b313800452b3"}, {"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "SYS_GEN__install_CRb_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "attrs": {"type": "CALL_RUNBOOK", "inarg_list": [], "runbook_reference": {"kind": "app_runbook", "name": "SYS_GEN__Runbook_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "b81bf28e-424c-4daa-9beb-8ae9a62855ba"}}, "timeout_secs": "", "type": "CALL_RUNBOOK", "variable_list": [], "uuid": "d4857a2b-32f6-471d-9e05-295df9e45c87"}, {"target_any_local_reference": {"kind": "app_service", "name": "code", "uuid": "0afc9748-88f7-8e8c-f7fe-b2b2cd6669e1"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "SYS_GEN__create_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "state": "ACTIVE", "attrs": {"type": "CALL_RUNBOOK", "inarg_list": [], "runbook_reference": {"kind": "app_runbook", "name": "eba78c76_runbook_cloned_1", "uuid": "e3ec95e5-6ede-c168-b629-600d771f5803"}}, "timeout_secs": "", "type": "CALL_RUNBOOK", "variable_list": [], "uuid": "55697484-6734-4bdd-afde-75e8c171697e"}, {"target_any_local_reference": {"kind": "app_service", "name": "code", "uuid": "0afc9748-88f7-8e8c-f7fe-b2b2cd6669e1"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "SYS_GEN__start_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "state": "ACTIVE", "attrs": {"type": "CALL_RUNBOOK", "inarg_list": [], "runbook_reference": {"kind": "app_runbook", "name": "3cedb317_runbook_cloned_1", "uuid": "aeedc907-4495-18e4-e7aa-93b781b90737"}}, "timeout_secs": "", "type": "CALL_RUNBOOK", "variable_list": [], "uuid": "d5751cd2-f72a-4f35-9fd6-104079471e54"}], "description": "", "name": "SYS_GEN__Runbook_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "SYS_GEN__Composite_DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "dcc6c9a0-ef16-406c-8960-b313800452b3"}, "message_list": [], "variable_list": [], "uuid": "daf9342b-98d9-4877-aee9-b322ed207462"}, "message_list": [], "name": "action_create"}, {"description": "System action for uninstalling an application", "type": "system", "uuid": "3255d19b-f2af-4d1f-a517-f91bf14bdc93", "state": "ACTIVE", "critical": true, "attrs": {}, "runbook": {"task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [{"kind": "app_task", "name": "PE_Uninstall_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "aa9bdd7f-4d40-4d9c-af4d-13091c066f70"}], "name": "SYS_GEN__DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "attrs": {"edges": [], "type": "DAG"}, "timeout_secs": "", "type": "DAG", "variable_list": [], "uuid": "92cd3b4b-fc1a-4dc3-9f17-6a5de9a2473e"}, {"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "PE_Uninstall_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "attrs": {"install_runbook": {"task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [{"kind": "app_task", "name": "PackageInstallTask", "uuid": "b24dd948-e65d-2c1e-52d8-c4d21d615274"}], "name": "6405be68_dag", "state": "ACTIVE", "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "uuid": "7ec268af-4e53-3c55-930a-740501d41cfb"}, {"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "PackageInstallTask", "state": "ACTIVE", "attrs": {"exit_status": [], "script": "#!/usr/bin/env python\nfrom cm_api.api_client import ApiResource\nfrom cm_api.endpoints.clusters import ApiCluster\nfrom cm_api.endpoints.clusters import create_cluster\nfrom cm_api.endpoints.parcels import ApiParcel\nfrom cm_api.endpoints.parcels import get_parcel\nfrom cm_api.endpoints.cms import ClouderaManager\nfrom cm_api.endpoints.services import ApiService, ApiServiceSetupInfo\nfrom cm_api.endpoints.services import create_service\nfrom cm_api.endpoints.types import ApiCommand, ApiRoleConfigGroupRef\nfrom cm_api.endpoints.role_config_groups import get_role_config_group\nfrom cm_api.endpoints.role_config_groups import ApiRoleConfigGroup\nfrom cm_api.endpoints.roles import ApiRole\nfrom time import sleep\n\nhost_username = 'root'\ncm_repo_url = None\ncdh_version = \"CDH5\" # also valid: \"CDH4\"\ncdh_version_number = \"5\"\n#240\nsleep(240)\n\nhostsIP = [ ]\nhostNames = [ ]\nhostNamesLong = [ ]\nthostNames = [ ]\nthostNames = \"@@{compute.name}@@\"\n#hostsIP = thostsIP.split(,)\nhostNames = thostNames.split(\",\")\n\n#for name in hostName:\n # hostNamesLong = hostName.append(name + '.rf9.local')\n\nfor name in hostNames:\n hostNamesLong.append(name + '.rf9.local')\n print hostNamesLong\n \nprint \"@@{code.cmip}@@\"\nprint \"@@{code.cmPassword}@@\"\n\napi = ApiResource(\"@@{code.cmip}@@\", '7180', \"@@{code.cmAdmin}@@\", \"@@{code.cmPassword}@@\", version=\"@@{code.APIversion}@@\")\n\n#get the CM instance\ncm = api.get_cluster(\"@@{code.clusterName}@@\")\ncm = ClouderaManager(api)\n\n\ncmd = cm.host_install(\"@@{code.hostAdmin}@@\", hostNamesLong, password=\"@@{code.hostPassword}@@\", cm_repo_url=cm_repo_url)\nprint \"Installing hosts. This might take a while.\"\nwhile cmd.success == None:\n sleep(15)\n cmd = cmd.fetch()\n\nif cmd.success != True:\n print \"cm_host_install failed: \" + cmd.resultMessage\n# exit(0)\nprint \"cm_host_install succeeded\"\n\ncluster = api.get_cluster(\"@@{code.clusterName}@@\")\n\nhostsadded = cluster.add_hosts(hostNamesLong)\n\nparcels_list = []\n# get and list all available parcels\nprint \"Available parcels:\"\nfor p in cluster.get_all_parcels():\n print p.version\n if p.version.startswith(\"@@{code.cmPackage}@@\") and p.product == \"CDH\":\n parcels_list.append(p)\n\nif len(parcels_list) == 0:\n print \"No \"\n print cdh_version\n print \" parcel found!\"\n exit(0)\n\ncdh_parcel = parcels_list[0]\n\nfor p in parcels_list:\n if p.version > cdh_parcel.version:\n cdh_parcel = p\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" downloaded\"\n\n# distribute the parcel\nprint \"Starting parcel distribution. This might take a while.\"\ncmd = cdh_parcel.start_distribution()\nif cmd.success != True:\n print \"Parcel distribution failed!\"\n exit(0)\n\nprint 'wait 4 min'\n# make sure the distribution finishes\n#while cdh_parcel.stage != \"DISTRIBUTED\":\nsleep(200)\n# cdh_parcel = get_parcel(api, cdh_parcel.product, '5.13.3-1.cdh5.13.3.p0.2', cluster_name)\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" distributed\"\n\n# activate the parcel\ncmd = cdh_parcel.activate()\nif cmd.success != True:\n print \"Parcel activation failed!\"\n exit(0)\n\n# make sure the activation finishes\n#while cdh_parcel.stage != \"ACTIVATED\":\n# cdh_parcel = get_parcel(api, cdh_parcel.product, '5.13.3-1.cdh5.13.3.p0.2', cluster_name)\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" activated\"\n\n\nyarn = cluster.get_service('yarn')\nprint yarn.name\nhostsid = api.get_all_hosts()\n#range = len(hostsid)\n#print 'range: ' + range\n\nfor current in hostsid:\n if current.rackId == '/default/default':\n print 'host name to add' + current.hostname\n currentshort1 = current.hostname\n currentshort = currentshort1.split(\".\", 1)\n short = currentshort[0]\n print short\n yarn.create_role(short, 'NODEMANAGER', current.hostId)\n\ncluster.restart(restart_only_stale_services=True, redeploy_client_configuration=True)\n\nexit(0)", "script_type": "sh", "type": "", "command_line_args": "", "login_credential_local_reference": {"kind": "app_credential", "name": "go", "uuid": "f356d2b6-2fac-453c-84ad-d91bc21bb1ba"}}, "timeout_secs": "", "type": "EXEC", "variable_list": [], "uuid": "b24dd948-e65d-2c1e-52d8-c4d21d615274"}], "description": "", "name": "7da2879a_runbook_cloned_1", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "6405be68_dag", "uuid": "7ec268af-4e53-3c55-930a-740501d41cfb"}, "message_list": [], "variable_list": [], "uuid": "2363d280-af23-b97a-557f-1e6a3d156fbb"}, "type": "", "uninstall_runbook": {"task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "de4520ca_dag", "state": "ACTIVE", "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "uuid": "aabae04b-08bb-466e-76a2-1fcfc3c816cd"}], "description": "", "name": "339b443d_runbook_cloned_1", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "de4520ca_dag", "uuid": "aabae04b-08bb-466e-76a2-1fcfc3c816cd"}, "message_list": [], "variable_list": [], "uuid": "ff185ae5-bc26-095b-6840-9fc957638621"}}, "timeout_secs": "", "type": "CUSTOM_PACKAGE_UNINSTALL", "variable_list": [], "uuid": "aa9bdd7f-4d40-4d9c-af4d-13091c066f70"}], "description": "", "name": "SYS_GEN__Runbook_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "SYS_GEN__DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "92cd3b4b-fc1a-4dc3-9f17-6a5de9a2473e"}, "message_list": [], "variable_list": [], "uuid": "3efda7af-c2c5-4527-91b8-0c58ae037827"}, "message_list": [], "name": "action_uninstall"}, {"description": "System action for deleting an application. Deletes physical machines as well", "type": "system", "uuid": "7ad1b2bf-b87c-4d3d-8da5-33abeb3cb757", "state": "ACTIVE", "critical": true, "attrs": {}, "runbook": {"task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [{"kind": "app_task", "name": "SYS_GEN__stop_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "5b2b1a62-4e83-4d1b-934b-862d89fc58f9"}, {"kind": "app_task", "name": "SYS_GEN__delete_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "2ce000b8-1296-4f5d-a8ae-8da96433a1cf"}, {"kind": "app_task", "name": "SYS_GEN__uninstall_CRb_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "1a05ced5-575f-4daf-ae7c-033403a0e77d"}], "name": "SYS_GEN__Composite_DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "attrs": {"edges": [{"from_task_reference": {"kind": "app_task", "name": "SYS_GEN__stop_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "5b2b1a62-4e83-4d1b-934b-862d89fc58f9"}, "edge_type": "inherent", "type": "", "uuid": "60c3c99d-672f-49af-b994-89bd9f92909e", "to_task_reference": {"kind": "app_task", "name": "SYS_GEN__delete_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "2ce000b8-1296-4f5d-a8ae-8da96433a1cf"}}, {"from_task_reference": {"kind": "app_task", "name": "SYS_GEN__delete_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "2ce000b8-1296-4f5d-a8ae-8da96433a1cf"}, "edge_type": "inherent", "type": "", "uuid": "be945022-5672-4d55-9a03-54584bb4b48f", "to_task_reference": {"kind": "app_task", "name": "SYS_GEN__uninstall_CRb_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "1a05ced5-575f-4daf-ae7c-033403a0e77d"}}], "type": "DAG"}, "timeout_secs": "", "type": "DAG", "variable_list": [], "uuid": "24022115-efdb-4ecf-b9b2-6275a9944739"}, {"target_any_local_reference": {"kind": "app_service", "name": "code", "uuid": "0afc9748-88f7-8e8c-f7fe-b2b2cd6669e1"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "SYS_GEN__stop_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "state": "ACTIVE", "attrs": {"type": "CALL_RUNBOOK", "inarg_list": [], "runbook_reference": {"kind": "app_runbook", "name": "5be55dea_runbook_cloned_1", "uuid": "aec97efe-ee30-0743-f90f-192698db865c"}}, "timeout_secs": "", "type": "CALL_RUNBOOK", "variable_list": [], "uuid": "5b2b1a62-4e83-4d1b-934b-862d89fc58f9"}, {"target_any_local_reference": {"kind": "app_service", "name": "code", "uuid": "0afc9748-88f7-8e8c-f7fe-b2b2cd6669e1"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "SYS_GEN__delete_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "state": "ACTIVE", "attrs": {"type": "CALL_RUNBOOK", "inarg_list": [], "runbook_reference": {"kind": "app_runbook", "name": "740b4861_runbook_cloned_1", "uuid": "3b9eb309-30d3-a2ae-137b-9d1552ea6038"}}, "timeout_secs": "", "type": "CALL_RUNBOOK", "variable_list": [], "uuid": "2ce000b8-1296-4f5d-a8ae-8da96433a1cf"}, {"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "SYS_GEN__uninstall_CRb_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "attrs": {"type": "CALL_RUNBOOK", "inarg_list": [], "runbook_reference": {"kind": "app_runbook", "name": "SYS_GEN__Runbook_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "3efda7af-c2c5-4527-91b8-0c58ae037827"}}, "timeout_secs": "", "type": "CALL_RUNBOOK", "variable_list": [], "uuid": "1a05ced5-575f-4daf-ae7c-033403a0e77d"}], "description": "", "name": "SYS_GEN__Runbook_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "SYS_GEN__Composite_DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "24022115-efdb-4ecf-b9b2-6275a9944739"}, "message_list": [], "variable_list": [], "uuid": "13878dbf-d8a6-46ae-b2ba-91b2d94c80b6"}, "message_list": [], "name": "action_delete"}, {"description": "System action for deleting an application. Does not delete physical machines", "type": "system", "uuid": "514bcd42-76cc-4588-aaca-d8c090d2b394", "state": "ACTIVE", "critical": true, "attrs": {}, "runbook": {"task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [{"kind": "app_task", "name": "SYS_GEN__soft_delete_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "4378066f-61a2-455f-bac8-42fded9ad756"}, {"kind": "app_task", "name": "SYS_GEN__Package_Element_Delete_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "c3dcc740-7e1c-40b9-9c5f-80243f049090"}], "name": "SYS_GEN__Composite_DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "attrs": {"edges": [{"from_task_reference": {"kind": "app_task", "name": "SYS_GEN__soft_delete_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "4378066f-61a2-455f-bac8-42fded9ad756"}, "edge_type": "inherent", "type": "", "uuid": "eaa03831-dedd-430b-a855-b2f31ce3a575", "to_task_reference": {"kind": "app_task", "name": "SYS_GEN__Package_Element_Delete_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "c3dcc740-7e1c-40b9-9c5f-80243f049090"}}], "type": "DAG"}, "timeout_secs": "", "type": "DAG", "variable_list": [], "uuid": "b0c7ab87-e166-4b64-8e34-ecff50898741"}, {"target_any_local_reference": {"kind": "app_service", "name": "code", "uuid": "0afc9748-88f7-8e8c-f7fe-b2b2cd6669e1"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "SYS_GEN__soft_delete_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "state": "ACTIVE", "attrs": {"type": "CALL_RUNBOOK", "inarg_list": [], "runbook_reference": {"kind": "app_runbook", "name": "SYS_GEN__Runbook_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "d9c0f039-0740-41ee-9d53-d5846fb8aa33"}}, "timeout_secs": "", "type": "CALL_RUNBOOK", "variable_list": [], "uuid": "4378066f-61a2-455f-bac8-42fded9ad756"}, {"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "SYS_GEN__Package_Element_Delete_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "attrs": {"type": ""}, "timeout_secs": "", "type": "SOFT_DELETE_ELEMENT", "variable_list": [], "uuid": "c3dcc740-7e1c-40b9-9c5f-80243f049090"}], "description": "", "name": "SYS_GEN__Runbook_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "SYS_GEN__Composite_DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "b0c7ab87-e166-4b64-8e34-ecff50898741"}, "message_list": [], "variable_list": [], "uuid": "a33c5528-ee5d-45c1-a762-1130af869bfd"}, "message_list": [], "name": "action_soft_delete"}, {"description": "System action for starting an application", "type": "system", "uuid": "a85a6591-bc55-4b22-9057-dc49a156d6d0", "state": "ACTIVE", "critical": true, "attrs": {}, "runbook": {"task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [{"kind": "app_task", "name": "SYS_GEN__start_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "4fbf8a01-298d-4363-8968-6f4b034ee440"}], "name": "SYS_GEN__Composite_DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "attrs": {"edges": [], "type": "DAG"}, "timeout_secs": "", "type": "DAG", "variable_list": [], "uuid": "c340dcbc-2937-40b0-82a9-e82c9ea49f06"}, {"target_any_local_reference": {"kind": "app_service", "name": "code", "uuid": "0afc9748-88f7-8e8c-f7fe-b2b2cd6669e1"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "SYS_GEN__start_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "state": "ACTIVE", "attrs": {"type": "CALL_RUNBOOK", "inarg_list": [], "runbook_reference": {"kind": "app_runbook", "name": "3cedb317_runbook_cloned_1", "uuid": "aeedc907-4495-18e4-e7aa-93b781b90737"}}, "timeout_secs": "", "type": "CALL_RUNBOOK", "variable_list": [], "uuid": "4fbf8a01-298d-4363-8968-6f4b034ee440"}], "description": "", "name": "SYS_GEN__Runbook_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "SYS_GEN__Composite_DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "c340dcbc-2937-40b0-82a9-e82c9ea49f06"}, "message_list": [], "variable_list": [], "uuid": "12813311-3bf3-486f-b59a-ec1087c2f54d"}, "message_list": [], "name": "action_start"}, {"description": "System action for stopping an application", "type": "system", "uuid": "cf8d0527-c638-4d40-8947-a8bf1cb1d051", "state": "ACTIVE", "critical": true, "attrs": {}, "runbook": {"task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [{"kind": "app_task", "name": "SYS_GEN__stop_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "uuid": "9626db53-15f7-4f28-83b0-70061870f14f"}], "name": "SYS_GEN__Composite_DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "attrs": {"edges": [], "type": "DAG"}, "timeout_secs": "", "type": "DAG", "variable_list": [], "uuid": "fd0d81b5-563f-46bc-b001-b3e967bf45a0"}, {"target_any_local_reference": {"kind": "app_service", "name": "code", "uuid": "0afc9748-88f7-8e8c-f7fe-b2b2cd6669e1"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "SYS_GEN__stop_CRb_Service_0afc9748_88f7_8e8c_f7fe_b2b2cd6669e1", "state": "ACTIVE", "attrs": {"type": "CALL_RUNBOOK", "inarg_list": [], "runbook_reference": {"kind": "app_runbook", "name": "5be55dea_runbook_cloned_1", "uuid": "aec97efe-ee30-0743-f90f-192698db865c"}}, "timeout_secs": "", "type": "CALL_RUNBOOK", "variable_list": [], "uuid": "9626db53-15f7-4f28-83b0-70061870f14f"}], "description": "", "name": "SYS_GEN__Runbook_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "SYS_GEN__Composite_DAG_Package_44bbdab6_b71e_8989_d19b_aeb8c940122c", "uuid": "fd0d81b5-563f-46bc-b001-b3e967bf45a0"}, "message_list": [], "variable_list": [], "uuid": "6715e54e-21f8-4206-9d27-bb4518c0a012"}, "message_list": [], "name": "action_stop"}], "type": "DEB", "service_local_reference_list": [{"kind": "app_service", "name": "code", "uuid": "0afc9748-88f7-8e8c-f7fe-b2b2cd6669e1"}], "name": "cloudera", "state": "ACTIVE", "version": "", "editables": {}, "message_list": [], "options": {"install_runbook": {"task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [{"kind": "app_task", "name": "PackageInstallTask", "uuid": "b24dd948-e65d-2c1e-52d8-c4d21d615274"}], "name": "6405be68_dag", "state": "ACTIVE", "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "uuid": "7ec268af-4e53-3c55-930a-740501d41cfb"}, {"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "PackageInstallTask", "state": "ACTIVE", "attrs": {"exit_status": [], "script": "#!/usr/bin/env python\nfrom cm_api.api_client import ApiResource\nfrom cm_api.endpoints.clusters import ApiCluster\nfrom cm_api.endpoints.clusters import create_cluster\nfrom cm_api.endpoints.parcels import ApiParcel\nfrom cm_api.endpoints.parcels import get_parcel\nfrom cm_api.endpoints.cms import ClouderaManager\nfrom cm_api.endpoints.services import ApiService, ApiServiceSetupInfo\nfrom cm_api.endpoints.services import create_service\nfrom cm_api.endpoints.types import ApiCommand, ApiRoleConfigGroupRef\nfrom cm_api.endpoints.role_config_groups import get_role_config_group\nfrom cm_api.endpoints.role_config_groups import ApiRoleConfigGroup\nfrom cm_api.endpoints.roles import ApiRole\nfrom time import sleep\n\nhost_username = 'root'\ncm_repo_url = None\ncdh_version = \"CDH5\" # also valid: \"CDH4\"\ncdh_version_number = \"5\"\n#240\nsleep(240)\n\nhostsIP = [ ]\nhostNames = [ ]\nhostNamesLong = [ ]\nthostNames = [ ]\nthostNames = \"@@{compute.name}@@\"\n#hostsIP = thostsIP.split(,)\nhostNames = thostNames.split(\",\")\n\n#for name in hostName:\n # hostNamesLong = hostName.append(name + '.rf9.local')\n\nfor name in hostNames:\n hostNamesLong.append(name + '.rf9.local')\n print hostNamesLong\n \nprint \"@@{code.cmip}@@\"\nprint \"@@{code.cmPassword}@@\"\n\napi = ApiResource(\"@@{code.cmip}@@\", '7180', \"@@{code.cmAdmin}@@\", \"@@{code.cmPassword}@@\", version=\"@@{code.APIversion}@@\")\n\n#get the CM instance\ncm = api.get_cluster(\"@@{code.clusterName}@@\")\ncm = ClouderaManager(api)\n\n\ncmd = cm.host_install(\"@@{code.hostAdmin}@@\", hostNamesLong, password=\"@@{code.hostPassword}@@\", cm_repo_url=cm_repo_url)\nprint \"Installing hosts. This might take a while.\"\nwhile cmd.success == None:\n sleep(15)\n cmd = cmd.fetch()\n\nif cmd.success != True:\n print \"cm_host_install failed: \" + cmd.resultMessage\n# exit(0)\nprint \"cm_host_install succeeded\"\n\ncluster = api.get_cluster(\"@@{code.clusterName}@@\")\n\nhostsadded = cluster.add_hosts(hostNamesLong)\n\nparcels_list = []\n# get and list all available parcels\nprint \"Available parcels:\"\nfor p in cluster.get_all_parcels():\n print p.version\n if p.version.startswith(\"@@{code.cmPackage}@@\") and p.product == \"CDH\":\n parcels_list.append(p)\n\nif len(parcels_list) == 0:\n print \"No \"\n print cdh_version\n print \" parcel found!\"\n exit(0)\n\ncdh_parcel = parcels_list[0]\n\nfor p in parcels_list:\n if p.version > cdh_parcel.version:\n cdh_parcel = p\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" downloaded\"\n\n# distribute the parcel\nprint \"Starting parcel distribution. This might take a while.\"\ncmd = cdh_parcel.start_distribution()\nif cmd.success != True:\n print \"Parcel distribution failed!\"\n exit(0)\n\nprint 'wait 4 min'\n# make sure the distribution finishes\n#while cdh_parcel.stage != \"DISTRIBUTED\":\nsleep(200)\n# cdh_parcel = get_parcel(api, cdh_parcel.product, '5.13.3-1.cdh5.13.3.p0.2', cluster_name)\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" distributed\"\n\n# activate the parcel\ncmd = cdh_parcel.activate()\nif cmd.success != True:\n print \"Parcel activation failed!\"\n exit(0)\n\n# make sure the activation finishes\n#while cdh_parcel.stage != \"ACTIVATED\":\n# cdh_parcel = get_parcel(api, cdh_parcel.product, '5.13.3-1.cdh5.13.3.p0.2', cluster_name)\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" activated\"\n\n\nyarn = cluster.get_service('yarn')\nprint yarn.name\nhostsid = api.get_all_hosts()\n#range = len(hostsid)\n#print 'range: ' + range\n\nfor current in hostsid:\n if current.rackId == '/default/default':\n print 'host name to add' + current.hostname\n currentshort1 = current.hostname\n currentshort = currentshort1.split(\".\", 1)\n short = currentshort[0]\n print short\n yarn.create_role(short, 'NODEMANAGER', current.hostId)\n\ncluster.restart(restart_only_stale_services=True, redeploy_client_configuration=True)\n\nexit(0)", "script_type": "sh", "type": "", "command_line_args": "", "login_credential_local_reference": {"kind": "app_credential", "name": "go", "uuid": "f356d2b6-2fac-453c-84ad-d91bc21bb1ba"}}, "timeout_secs": "", "type": "EXEC", "variable_list": [], "uuid": "b24dd948-e65d-2c1e-52d8-c4d21d615274"}], "description": "", "name": "7da2879a_runbook_cloned_1", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "6405be68_dag", "uuid": "7ec268af-4e53-3c55-930a-740501d41cfb"}, "message_list": [], "variable_list": [], "uuid": "2363d280-af23-b97a-557f-1e6a3d156fbb"}, "type": "", "uninstall_runbook": {"task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera", "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}, "description": "", "message_list": [], "child_tasks_local_reference_list": [], "name": "de4520ca_dag", "state": "ACTIVE", "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "uuid": "aabae04b-08bb-466e-76a2-1fcfc3c816cd"}], "description": "", "name": "339b443d_runbook_cloned_1", "state": "ACTIVE", "main_task_local_reference": {"kind": "app_task", "name": "de4520ca_dag", "uuid": "aabae04b-08bb-466e-76a2-1fcfc3c816cd"}, "message_list": [], "variable_list": [], "uuid": "ff185ae5-bc26-095b-6840-9fc957638621"}}, "variable_list": [], "uuid": "44bbdab6-b71e-8989-d19b-aeb8c940122c"}]}, "name": "computeexpand"}, "spec": {"description": "", "resources": {"service_definition_list": [{"singleton": false, "name": "compute", "action_list": [{"critical": false, "type": "system", "description": "System action for creating an application", "name": "action_create", "runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_service", "name": "compute"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "b3c287b5_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "b3c287b5_dag"}, "name": "eba78c76_runbook"}}, {"critical": false, "type": "system", "description": "System action for deleting an application. Deletes physical machines as well", "name": "action_delete", "runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_service", "name": "compute"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "1744848a_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "1744848a_dag"}, "name": "740b4861_runbook"}}, {"critical": false, "type": "system", "description": "System action for starting an application", "name": "action_start", "runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_service", "name": "compute"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "87c10021_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "87c10021_dag"}, "name": "3cedb317_runbook"}}, {"critical": false, "type": "system", "description": "System action for stopping an application", "name": "action_stop", "runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_service", "name": "compute"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "c68f8d5e_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "c68f8d5e_dag"}, "name": "5be55dea_runbook"}}, {"critical": false, "type": "system", "description": "System action for restarting an application", "name": "action_restart", "runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_service", "name": "compute"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "5869d3a7_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "5869d3a7_dag"}, "name": "4ebd0004_runbook"}}], "description": "", "port_list": [], "tier": "", "variable_list": [], "depends_on_list": []}, {"singleton": false, "name": "code", "action_list": [{"critical": false, "type": "system", "description": "System action for creating an application", "name": "action_create", "runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_service", "name": "code"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "b3c287b5_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "b3c287b5_dag"}, "name": "eba78c76_runbook_cloned_1"}}, {"critical": false, "type": "system", "description": "System action for deleting an application. Deletes physical machines as well", "name": "action_delete", "runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_service", "name": "code"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "1744848a_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "1744848a_dag"}, "name": "740b4861_runbook_cloned_1"}}, {"critical": false, "type": "system", "description": "System action for starting an application", "name": "action_start", "runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_service", "name": "code"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "87c10021_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "87c10021_dag"}, "name": "3cedb317_runbook_cloned_1"}}, {"critical": false, "type": "system", "description": "System action for stopping an application", "name": "action_stop", "runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_service", "name": "code"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "c68f8d5e_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "c68f8d5e_dag"}, "name": "5be55dea_runbook_cloned_1"}}, {"critical": false, "type": "system", "description": "System action for restarting an application", "name": "action_restart", "runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_service", "name": "code"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "5869d3a7_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "5869d3a7_dag"}, "name": "4ebd0004_runbook_cloned_1"}}], "description": "", "port_list": [], "tier": "", "variable_list": [{"val_type": "STRING", "description": "", "value": "admin", "label": "", "attrs": {"type": ""}, "type": "LOCAL", "name": "cmAdmin"}, {"val_type": "STRING", "description": "", "value": "10.15.80.180", "label": "", "attrs": {"type": ""}, "type": "LOCAL", "name": "cmip"}, {"val_type": "STRING", "description": "", "value": "", "label": "", "attrs": {"is_secret_modified": false, "secret_reference": {}, "type": ""}, "type": "SECRET", "name": "cmPassword"}, {"val_type": "STRING", "description": "", "value": "18", "label": "", "attrs": {"type": ""}, "type": "LOCAL", "name": "APIversion"}, {"val_type": "STRING", "description": "", "value": "Cluster 1", "label": "", "attrs": {"type": ""}, "type": "LOCAL", "name": "clusterName"}, {"val_type": "STRING", "description": "", "value": "5.13.3-1.cdh5.13.3", "label": "", "attrs": {"type": ""}, "type": "LOCAL", "name": "cmPackage"}, {"val_type": "STRING", "description": "", "value": "", "label": "", "attrs": {"is_secret_modified": false, "secret_reference": {}, "type": ""}, "type": "SECRET", "name": "hostPassword"}, {"val_type": "STRING", "description": "", "value": "root", "label": "", "attrs": {"type": ""}, "type": "LOCAL", "name": "hostAdmin"}], "depends_on_list": []}], "substrate_definition_list": [{"description": "", "action_list": [], "readiness_probe": {"connection_type": "SSH", "address": "@@{platform.status.resources.nic_list[0].ip_endpoint_list[0].ip}@@", "disable_readiness_probe": false, "timeout_secs": "60", "connection_port": 22}, "editables": {"create_spec": {"resources": {}}}, "os_type": "Linux", "type": "AHV_VM", "create_spec": {"backup_policy": null, "type": "", "name": "yarn@@{calm_application_name}@@-@@{calm_random}@@", "resources": {"hardware_clock_timezone": "", "type": "", "boot_config": null, "power_state": "ON", "guest_tools": null}, "availability_zone_reference": null}, "variable_list": [], "name": "yarn"}, {"description": "", "action_list": [], "readiness_probe": {"connection_type": "SSH", "address": "@@{platform.status.resources.nic_list[0].ip_endpoint_list[0].ip}@@", "disable_readiness_probe": false, "timeout_secs": "60", "connection_port": 22}, "editables": {"create_spec": {"resources": {}}}, "os_type": "Linux", "type": "AHV_VM", "create_spec": {"backup_policy": null, "type": "", "name": "run@@{calm_application_name}@@-@@{calm_random}@@", "resources": {"hardware_clock_timezone": "", "type": "", "boot_config": {"boot_device": {"type": "", "disk_address": {"adapter_type": "SCSI", "device_index": 0, "type": ""}}, "type": "", "mac_address": ""}, "power_state": "ON", "guest_tools": null}, "availability_zone_reference": null}, "variable_list": [], "name": "runner"}], "credential_definition_list": [{"username": "root", "secret": {"attrs": {"is_secret_modified": false, "secret_reference": {}}}, "type": "PASSWORD", "description": "", "name": "go"}], "package_definition_list": [{"description": "", "action_list": [], "service_local_reference_list": [{"kind": "app_service", "name": "compute"}], "version": "", "type": "DEB", "options": {"install_runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "deployCPUa"}, "description": "", "child_tasks_local_reference_list": [{"kind": "app_task", "name": "PackageInstallTask"}], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "6405be68_dag"}, {"target_any_local_reference": {"kind": "app_package", "name": "deployCPUa"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"exit_status": [], "script": "#!/usr/bin/env bash\n\nhostnamectl set-hostname @@{name}@@.rf9.local\n\nsystemctl restart network.service", "script_type": "sh", "type": "", "command_line_args": "", "login_credential_local_reference": {"kind": "app_credential", "name": "go"}}, "timeout_secs": "", "type": "EXEC", "variable_list": [], "name": "PackageInstallTask"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "6405be68_dag"}, "name": "7da2879a_runbook"}, "type": "", "uninstall_runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "deployCPUa"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "de4520ca_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "de4520ca_dag"}, "name": "339b443d_runbook"}}, "variable_list": [], "name": "deployCPUa"}, {"description": "", "action_list": [], "service_local_reference_list": [{"kind": "app_service", "name": "code"}], "version": "", "type": "DEB", "options": {"install_runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera"}, "description": "", "child_tasks_local_reference_list": [{"kind": "app_task", "name": "PackageInstallTask"}], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "6405be68_dag"}, {"target_any_local_reference": {"kind": "app_package", "name": "cloudera"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"exit_status": [], "script": "#!/usr/bin/env python\nfrom cm_api.api_client import ApiResource\nfrom cm_api.endpoints.clusters import ApiCluster\nfrom cm_api.endpoints.clusters import create_cluster\nfrom cm_api.endpoints.parcels import ApiParcel\nfrom cm_api.endpoints.parcels import get_parcel\nfrom cm_api.endpoints.cms import ClouderaManager\nfrom cm_api.endpoints.services import ApiService, ApiServiceSetupInfo\nfrom cm_api.endpoints.services import create_service\nfrom cm_api.endpoints.types import ApiCommand, ApiRoleConfigGroupRef\nfrom cm_api.endpoints.role_config_groups import get_role_config_group\nfrom cm_api.endpoints.role_config_groups import ApiRoleConfigGroup\nfrom cm_api.endpoints.roles import ApiRole\nfrom time import sleep\n\nhost_username = 'root'\ncm_repo_url = None\ncdh_version = \"CDH5\" # also valid: \"CDH4\"\ncdh_version_number = \"5\"\n#240\nsleep(240)\n\nhostsIP = [ ]\nhostNames = [ ]\nhostNamesLong = [ ]\nthostNames = [ ]\nthostNames = \"@@{compute.name}@@\"\n#hostsIP = thostsIP.split(,)\nhostNames = thostNames.split(\",\")\n\n#for name in hostName:\n # hostNamesLong = hostName.append(name + '.rf9.local')\n\nfor name in hostNames:\n hostNamesLong.append(name + '.rf9.local')\n print hostNamesLong\n \nprint \"@@{code.cmip}@@\"\nprint \"@@{code.cmPassword}@@\"\n\napi = ApiResource(\"@@{code.cmip}@@\", '7180', \"@@{code.cmAdmin}@@\", \"@@{code.cmPassword}@@\", version=\"@@{code.APIversion}@@\")\n\n#get the CM instance\ncm = api.get_cluster(\"@@{code.clusterName}@@\")\ncm = ClouderaManager(api)\n\n\ncmd = cm.host_install(\"@@{code.hostAdmin}@@\", hostNamesLong, password=\"@@{code.hostPassword}@@\", cm_repo_url=cm_repo_url)\nprint \"Installing hosts. This might take a while.\"\nwhile cmd.success == None:\n sleep(15)\n cmd = cmd.fetch()\n\nif cmd.success != True:\n print \"cm_host_install failed: \" + cmd.resultMessage\n# exit(0)\nprint \"cm_host_install succeeded\"\n\ncluster = api.get_cluster(\"@@{code.clusterName}@@\")\n\nhostsadded = cluster.add_hosts(hostNamesLong)\n\nparcels_list = []\n# get and list all available parcels\nprint \"Available parcels:\"\nfor p in cluster.get_all_parcels():\n print p.version\n if p.version.startswith(\"@@{code.cmPackage}@@\") and p.product == \"CDH\":\n parcels_list.append(p)\n\nif len(parcels_list) == 0:\n print \"No \"\n print cdh_version\n print \" parcel found!\"\n exit(0)\n\ncdh_parcel = parcels_list[0]\n\nfor p in parcels_list:\n if p.version > cdh_parcel.version:\n cdh_parcel = p\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" downloaded\"\n\n# distribute the parcel\nprint \"Starting parcel distribution. This might take a while.\"\ncmd = cdh_parcel.start_distribution()\nif cmd.success != True:\n print \"Parcel distribution failed!\"\n exit(0)\n\nprint 'wait 4 min'\n# make sure the distribution finishes\n#while cdh_parcel.stage != \"DISTRIBUTED\":\nsleep(200)\n# cdh_parcel = get_parcel(api, cdh_parcel.product, '5.13.3-1.cdh5.13.3.p0.2', cluster_name)\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" distributed\"\n\n# activate the parcel\ncmd = cdh_parcel.activate()\nif cmd.success != True:\n print \"Parcel activation failed!\"\n exit(0)\n\n# make sure the activation finishes\n#while cdh_parcel.stage != \"ACTIVATED\":\n# cdh_parcel = get_parcel(api, cdh_parcel.product, '5.13.3-1.cdh5.13.3.p0.2', cluster_name)\n\nprint cdh_parcel.product + ' ' + cdh_parcel.version + \" activated\"\n\n\nyarn = cluster.get_service('yarn')\nprint yarn.name\nhostsid = api.get_all_hosts()\n#range = len(hostsid)\n#print 'range: ' + range\n\nfor current in hostsid:\n if current.rackId == '/default/default':\n print 'host name to add' + current.hostname\n currentshort1 = current.hostname\n currentshort = currentshort1.split(\".\", 1)\n short = currentshort[0]\n print short\n yarn.create_role(short, 'NODEMANAGER', current.hostId)\n\ncluster.restart(restart_only_stale_services=True, redeploy_client_configuration=True)\n\nexit(0)", "script_type": "sh", "type": "", "command_line_args": "", "login_credential_local_reference": {"kind": "app_credential", "name": "go"}}, "timeout_secs": "", "type": "EXEC", "variable_list": [], "name": "PackageInstallTask"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "6405be68_dag"}, "name": "7da2879a_runbook_cloned_1"}, "type": "", "uninstall_runbook": {"variable_list": [], "task_definition_list": [{"target_any_local_reference": {"kind": "app_package", "name": "cloudera"}, "description": "", "child_tasks_local_reference_list": [], "attrs": {"edges": [], "type": ""}, "timeout_secs": "", "type": "DAG", "variable_list": [], "name": "de4520ca_dag"}], "description": "", "main_task_local_reference": {"kind": "app_task", "name": "de4520ca_dag"}, "name": "339b443d_runbook_cloned_1"}}, "variable_list": [], "name": "cloudera"}], "app_profile_list": [{"deployment_create_list": [{"name": "76df5aad_deployment", "action_list": [], "package_local_reference_list": [{"kind": "app_package", "name": "deployCPUa"}], "editables": {"min_replicas": true}, "max_replicas": "1", "substrate_local_reference": {"kind": "app_substrate", "name": "yarn"}, "min_replicas": "1", "variable_list": [], "description": ""}, {"description": "", "action_list": [], "package_local_reference_list": [{"kind": "app_package", "name": "cloudera"}], "max_replicas": "1", "substrate_local_reference": {"kind": "app_substrate", "name": "runner"}, "min_replicas": "1", "variable_list": [], "name": "76df5aad_deployment_cloned_1"}], "variable_list": [], "description": "", "action_list": [], "name": "Default1"}], "default_credential_local_reference": {"kind": "app_credential", "name": "go"}, "client_attrs": {"None": {"y": 450, "x": 810}, "code": {"y": 380, "x": 760}, "compute": {"y": 380, "x": 580}}}, "name": "computeexpand"}, "api_version": "3.0", "metadata": {"last_update_time": "1526503361119604", "creation_time": "1520027297574039", "kind": "blueprint", "spec_version": 85, "name": "computeexpand"}}

    Apr
    23

    AFS Cluster Network Change (Re-IP AFS) : Because Things Change

    You can now change the managed/unmanaged network settings of the file server FSVM cluster to support moving the cluster from one datacenter to another, updating VLAN settings, or other use cases where you need to change network information. For unmanaged networks, you can change the FSVM cluster virtual IP address or IP address of each FSVM. Now that its in the GUI, less human error and more time for coffee.

    Apr
    16

    AFS 3.0 Brings NFS for the Busy Developer

    AFS 3.0 brings NFS v4 Support! This is the rocket ship that your software builds needed! No longer are your build piplines stuck with the lonely power of a single NAS head servicing your workloads.

    AFS with NFS support enables you to manage a collection of NFS exports distributed across multiple file server VMs (FSVMs, think NAS head). With NFS, users can now use Linux and UNIX clients with AFS. This feature is also hypervisor agnostic.

    AFS supports two types of NFS exports:

    Distributed. A distributed export (“sharded”) means the data is spread across all FSVMs to help improve performance and resiliency. A distributed export can be used for any application. It is distributed at the top-level directories and does not have files at the root of the export. If you give a developer one share and each build goes into the share as top-level directory watch out, you might not have time for coffee.

    1 Share, multiple top-level directories, multiple NAS heads.

    Non-distributed. A non-distributed export (“non-sharded”) means all data is contained in a single FSVM. A non-distributed export is used for any purpose that does not require a distributed structure. If you have 10’s or 1000’s of exports they will be placed among all of the FSVM/NAS heads!

    Best of all, one click upgrades and the Nutanix ease of use makes this a slam dunk to deploy and maintain.