Apr
16

AFS 3.0 Brings NFS for the Busy Developer

AFS 3.0 brings NFS v4 Support! This is the rocket ship that your software builds needed! No longer are your build piplines stuck with the lonely power of a single NAS head servicing your workloads.

AFS with NFS support enables you to manage a collection of NFS exports distributed across multiple file server VMs (FSVMs, think NAS head). With NFS, users can now use Linux and UNIX clients with AFS. This feature is also hypervisor agnostic.

AFS supports two types of NFS exports:

Distributed. A distributed export (“sharded”) means the data is spread across all FSVMs to help improve performance and resiliency. A distributed export can be used for any application. It is distributed at the top-level directories and does not have files at the root of the export. If you give a developer one share and each build goes into the share as top-level directory watch out, you might not have time for coffee.

1 Share, multiple top-level directories, multiple NAS heads.

Non-distributed. A non-distributed export (“non-sharded”) means all data is contained in a single FSVM. A non-distributed export is used for any purpose that does not require a distributed structure. If you have 10’s or 1000’s of exports they will be placed among all of the FSVM/NAS heads!

Best of all, one click upgrades and the Nutanix ease of use makes this a slam dunk to deploy and maintain.

Mar
12

Docker Datacenter beta and Nutanix Kubernetes Volume Plugin

I got to test Persistent Volume Claims with DDC 3.0 beta from Docker and the Nutanix Kubernetes Volume Plugin. The automatic clean up was pretty interesting and slightly scary if you didn’t know about it!

https://next.nutanix.com/blog-40/nutanix-kubernetes-volume-plugin-for-on-demand-choice-27598

Jan
28

On-Prem or Cloud Hosted Kubernetes?

Should I host by Kubernetes cluster in the Cloud or keep it with in my own data center? Nutanix Calm can deploy on-prem or to a Cloud provider so what should I do?

Some questions I was thinking about when talking to peers. I would love this list to be builted overtime.

Connectivity
Do you require a connection back to the corporate network?
Where will your logging and management be located? Sending data back to a corp site needs to be factored into the cost.

Data Dependency
What data sources are required for your app?
How much latency can you tolerate?

Control
How much control do you for operations?
Can you find a on-prem solution that provide the updates and upgrades. Does the solution/infrastructure it sits on painless to also upgrade/patch
Stackdriver is tied to account on GCP. You can’t share data across accounts.
Do you have to intergate other on-prem monitoring or logging solutions.

Last updated January 28, 2018

    Dec
    20

    Docker Swarm with Nutanix Calm

    Review -> What is Nutanix CALM?

    Nutanix Calm provides a set of pre-seeded application blueprints that are available to you for consumption.

    Docker Swarm is a clustering and scheduling tool for Docker containers. Lots of hype with Kubernetes right now and rightly so but Swarm is a great tool and still getting better. One of the blueprints available with Calm is Docker Swarm. With Swarm, IT administrators and developers can establish and manage a cluster of Docker nodes as a single virtual system. Swarm mode also exists natively for Docker Engine, the layer between the OS and container images. Swarm mode integrates the orchestration capabilities of Docker Swarm into Docker Engine. For AHV, by default blueprint creates 3 Master VMs with 2 Core, 4GB RAM, Root disk – 10GB, and Data Disk 3x10GB. For AWS, by default blueprint create 3 Slave VMs t2.medium, and Data Disk 3x10GB.


    Installed Version- Docker - 17.09.0.ce

    Variables

    DOCKER_VERSION - (Mandatory) Docker version default.
    INSTANCE_PUBLIC_KEY - (Mandatory) Instance public key (only for AHV).
    Click Marketplace tab.
    Click the Docker Swarm blueprint application.
    Click Launch.
    The blueprint application launch page is displayed.

    Enter a name for the application in the Name of the Application field. For the application blueprint naming conventions, see Launching an Application Blueprint.
    Select the Application profile.
    If the application profile is Nutanix then do the following.
    (Optional) Change the VM name.
    (Optional) Change the number of vCPUs and RAM.
    Select the NIC from the drop-down menu.
    Download the CentOS 7 from the repository.
    Enter the private key.
    If the application profile is AWS then do the following.
    (Optional) Change the VM name.
    Select the instance type.
    Select a CentOS 7 image as per the region and AZ.
    Select the VPC and subnet.
    Ensure that the security groups have access of ICMP port so that master and slave nodes can ping each other.

    Select the SSH keys.
    Repeat the above steps for docker slave services.

      Dec
      12

      Running IT: Docker and Cilium for Enterprise Network Security for Micro-Services

      Well I think 40 min is about as long as I can last watching a IT related video while running after that I need music! This time I watched another video from DockerCon, Cilium – Kernel Native Security & DDOS Mitigation for Microservices with BPF

      Skip to 7:23: The quick overview of the presentation is that managing IP Tables to lock down micro-services isn’t going to scale and will be almost impossible to manage. Cilium is open source software for providing and transparently securing network connectivity and load balancing between application workloads such as application containers or processes. Cilium operates at Layer 3/4 to provide traditional networking and security services as well as Layer 7 to protect and secure use of modern application protocols such as HTTP, gRPC and Kafka. BPF is used a lot of the big web-scale properties like Facebook and Netflix to secure their environment and to provide troubleshooting. Like anything with a lot of options there is a lot of ways to shoot yourself in the foot so Cilium provides the wrapper to get it easily deployed and configured.

      The presentation uses that example of locking down a Kafka cluster via layer 7 instead of having the whole API left wind open which would happen if your were only using IP tables. Kafka is used for building real-time pipelines and streaming apps. Kafka is horizontally scalable and fault-tolerant so it’s a good choice to run it in docker. Kakfa is used by 1/3 of Fortune 500 companies.

      Cilium Architecture

      Cilium Integrates with:

      Docker
      Kubernetes
      Mesos

      Cilium runs as a agent on every host.
      Cilium can provide policy for Host to Docker micro-service and even between two containers on the same host.

      The demo didn’t pan out but the 2nd half of the presentation talks about Cilium using BPF with XDP. XDP is a further step in evolution and enables to run a specific flavor of BPF programs from the network driver with direct access to the packet’s DMA buffer. This is, by definition, the earliest possible point in the software stack, where programs can be attached to in order to allow for a programmable, high performance packet processor in the Linux kernel networking data path.

      Since XDP can happen earlier on at the nic versus iptables with ipset, CPU can be saved, rules load faster and latency under load is a lot better with XDP.

      Nov
      27

      Running IT: Using Docker to Scale Operational Intelligence at Splunk

      I am trying to get back into running and would like to complete a marathon at some point but I am taking it slow. Last time I tired, I got up to 36 KM but knees and feet didn’t fare so well. With that being said I am going to have some time on the treadmill and elliptical and one way I can be of service is to tell you the important parts of video’s I watch and hopefully give you back some time as well. The first video I picked was Using Docker to Scale Operational Intelligence at Splunk from Dockercon 17 Europe.

      I was kinda hoping for Splunk and Docker Integration but it was more about how Splunk was using Docker. Interestingly Splunk does have a good use case for needing both Windows and Linux nodes for testing. When I first heard that Docker could have both Linux and Windows hosts in the same Swarm cluster I thought that was cool but didn’t really know how much it would be used.

      Skip to 21:33
      – First half is mainly review of Docker EE and why Splunk picked Docker. I am sure most have heard similar stories. There is mention of RBAC for Nodes which allows secure multi-tenancy across multiple teams through node-based isolation and segregation. At the time of the video Splunk wasn’t using it but would have made life easier.

      Interesting Notes for the Session

      Started with a Bare-metal test lab of 150 nodes using UCP. Now running over 600 servers.

      Splunk 7 was a new feature, Metrics. Metrics is a for system administrators and IT tools engineers that focuses on collecting, investigating, monitoring, and sharing metrics from your technology infrastructure, security systems, and business applications in real time. Splunk is using Collectd to get data into Splunk and also grabs the logs and search it from the same interface.

      Splunk using 1 container per server for performance testing.

      Or test/dev testing Splunk uses 20-30 containers per server.

      Running a complicated system to make sure performance and test/dev containers don’t intermix. Splunk is hoping to use the new RBAC for nodes and the CPU/Memory reservations to clean up the CI workflow.

      Moving away for Jenkins for CI. Using UCP to move away from agents to run over 2,000 concurrent jobs.

      Oct
      17

      My Thoughts On The Kubernetes Support For Docker EE

      Sep
      25

      Maximize Your ROI and Agility with Big Data #Nutanix #Docker #BlueData

      Separate out your data from your compute for more agility.

      The datanode is what is used to build out the HDFS. Typically the the dataNode and the nodeManager are co-located on the same host whether its physical or virtual. The NodeManager is responsible for launching and managing containers that are scheduled from the Resource Manager. On Nutanix if you virtualize the dataNode and the nodeManager on separate virtual machines you have the opportunity to increase your agility. The agility comes from the ability to use your resources to the max of your capacity at all times. When the the cluster isn’t in use or as busy, other systems have the opportunity to use the resources. You can shut down the NodeManager since they’re not responsible for persisting data and make the the CPU and memory available for another project like Spark or maybe a new machine-learning program someone wants to test out.

      Hold the phone! What about data locality? You are correct performance is going to take a hit. Performance may drop from up to 15% from the standard way but if your system is only busy 30% of time it might be more that worth it. Let’s say a job takes 60 minutes to complete. Using this new model of separating out compute and storage, the job may now take 70 minutes to complete. Is the extra 10 minutes worth the agility to use your hardware for other projects? I think so but that is going to depend on your business requirements of course.

      On the data locality side, the datenode still gets to benefit from reading locally. It’s data path on the network isn’t going to cause more stress so that’s a plus. Also the nodeManager is busy writing all of the temporary and shuffle data locally so that is also not going to cause any additional stress compared to having the nodeManager write to a remote shared storage device. Also in some cases the NodeManager will still talk to the local datanode over the local hypervisor switch.

      If your after some real flexibility you could look at using BlueData to run Docker containers along side the dataNodes. BlueData will take over for the nodeManager essentially. Install some CentOS VMs that fit inside the hosts NUMA node and install BlueData. BlueData can help with QofS for different tenants, allow you to run different versions of Hadoop distros, Spark, Kafka and son on without blowing out your data requirements. BlueData also helps to maximize the remote connection between the containers and HDFS distro of choice.

      If your after more agility, avoiding separate hardware for projects, getting better ROI for systems that run only weekly, monthly, quarterly or better testing methodologies this may be the right architecture for you to try out.

      Sep
      07

      Windows Get Some Love with #Docker EE 17.06

      With the new release of Docker 17.06 EE Windows containers gets lots of added features. First up is the ability to run Windows and Linux worker nodes in the same same cluster. This is great because you have centralized security and logging across your whole environment. Your .NET and Java teams can live in peace to consolidate your infrastructure instead of spinning of separate environments.

      Continuously scanning for vulnerabilities in Windows images was added if your have Advanced EE license. Not only does it scan images it will also alert when new vulnerabilities are found in existing images.

      Bringing everything together you can use the same overlay networks to connect your application in the case of SQL server and web servers running on Linux. Your developers can create a single compose file covering both SQL and web severs.

      Other New Windows related features in Docker 17.06:

      Windows Server 2016 support
      Windows 10586 is marked as deprecated; it will not be supported going forward in stable releases
      Integration with Docker Cloud, with the ability to control remote Swarms from the local command line interface (CLI) and view your repositories
      Unified login between the Docker CLI and Docker Hub, Docker Cloud.
      Sharing a drive can be done on demand, the first time a mount is requested
      Add an experimental DNS name for the host: docker.for.win.localhost
      Support for client (i.e. “login”) certificates for authenticating registry access (fixes docker/for-win#569)
      New installer experience

      Sep
      04

      Multi-stage build support in #Docker EE 17.06

      New in Docker EE 17.06 is the ability to have multi-stage builds. This is important because you can now just grab the files(artifacts) you need for the next stage of your build and keep your builds small which leads to faster build times. This change allows you to have mutiple from arguments in your docker file.

      Devs have optional give a name the build stage. Then afterward this name can be used in COPY –from=name src dest and FROM name. If a build stage is defined with that name it takes precedence in these commands, if it is not found, an image with that name is attempted to be used instead.

      FROM node AS test-env
      ADD ./ /app
      WORKDIR /app
      RUN npm install
      RUN npm run build

      FROM nginx AS prod
      COPY --from=test-env /app /var/www/html

      You can run subsets of the dockerfile to get more use of out your work. If you wanted to only run the test-env section you can add a target to the docker build command with –target test-env