Tech News - DevOps

Chaos engineering has been a trend to watch for the last 12 months, but it is yet to really capture the imagination of the global software industry. It remains a pretty specialised discipline confined to the most forward thinking companies who depend on extensive distributed systems. However, that could all be about to change thanks to Gremlin who have today announced the launch of Gremlin Free. Gremlin Free is a tool that allows software, infrastructure and DevOps engineers to perform shutdown and CPU attacks on their infrastructure in a safe and controlled way using a neat and easy to use UI. In a blog post published on the Gremlin site today, Lorne Kligerman, Director of Product, said "we believe the industry has answered why do chaos engineering, and has begun asking how do I begin practicing Chaos Engineering in order to significantly increase the reliability and resiliency of our systems to provide the best user experience possible." Read next: How Gremlin is making chaos engineering accessible [Interview] What is Gremlin free? Gremlin Free is based on Netflix's Chaos Monkey tool. Chaos Monkey is the tool that gave rise to chaos engineering way back in 2011 when the streaming platform first moved to AWS. It let Netflix engineers "randomly shut down compute instances," which became a useful tactic for stress testing the reliability and resilience of its new microservices architecture. What can you do with Gremlin Free? There are two attacks you can do with Gremlin Free: Shutdown and CPU. As the name indicates, Shutdown lets you take down (or reboot) multiple hosts or containers. CPU attacks simply allow you to cause spikes in CPU usage to monitor its impact on your infrastructure. Both attacks can help teams identify pain points within their infrastructure, and ultimately form the foundations of an engineering strategy that relies heavily on the principles of chaos engineering. Why Gremlin Free now? Gremlin cites data from Gartner that underlines just how expensive downtime can be: according to Gartner, eCommerce companies can lose an average of $5,600 per minute, with that figure stretching even bigger for the planet's leading eCommerce businesses. However, despite the cost of downtime making a clear argument for chaos engineering's value, its adoption isn't widespread - certainly not as widespread as Gremlin believe it should be. Kligerman said "It's still a new concept to most engineering teams, so we wanted to offer a free version of our software that helps them become more familiar with chaos engineering - from both a tooling and culture perspective." If you're interested in trying chaos engineering, sign up for Gremlin Free here.

The LXD team released version 3.11 of LXD, its open source container management extension for Linux Containers (LXC), earlier this week. LXD 3.11 explores new features, minor improvements, and bugfixes. LXD or ‘ Linux Daemon’ system container manager provides users with an experience similar to virtual machines. It is written in Go and helps improve the existing LXC features to build and manage Linux containers. New Features in LXD 3.11 Configurable snapshot expiry at creation time: LXD 3.11 allows users to set an expiry during the snapshot creation time. Earlier, it was a hassle to manually create snapshots and edit them to modify their expiry. To change the expiry at the API level, you can set the exact timestamp to null that will make a persistent snapshot despite any configured auto-expiry. Progress reporting for publish operations: Progress information is now displayed to the user in LXD 3.11 when running lxc publish against a container or snapshot. This is similar to image transfers and container migrations. Improvements Minor improvements have been made to how candid authentication feature gets handled by the CLI in LXD 3.11. Per-remote authentication cookies: Now every remote consist of its own “cookie jar”. Also, LXD’s behavior is now always identical in LXD 3.11 when adding remotes. In prior releases, a shared “cookie jar” was being used for all remotes which would lead to inconsistent behaviors. Candid preferred over TLS for new remotes: In LXD 3.11, while using LXC remote add to add in a new remote, candid will be used for TLS authentication in case that remote supports candid. Also, authentication type can always be overridden using --auth-type. Remote list can now show Candid domain: The remote list can now indicate what Candid domain is used in LXD 3.11. Bug Fixes Goroutine leak has been fixed in ExecContainer. The “client: fix goroutine leak in ExecContainer” has been reverted. rest-api.md formatting has been updated. Translations from weblate have also been updated. Error handling in execIfAliases has been improved. Duplicate scheduled snapshots have been fixed. failing backup import has been fixed. Test case that covers the image sync scenario for the joined node has been updated. For a complete list of changes, check out the official LXD 3.11 release notes. LXD 3.8 released with automated container snapshots, ZFS compression support and more! Linux 4.20 kernel slower than its previous stable releases, Spectre flaw to be blamed, according to Phoronix An update on Bcachefs- the “next generation Linux filesystem”

Yesterday, at Amazon re:Invent, Aviatrix, a tool that helps users manage cloud deployments, announced and demonstrated Aviatrix Orchestrator. This new feature will make connecting multiple networks much easier. Essentially, it unifies the management of both AWS native networking services and Aviatrix services via a single management console. How does Aviatrix Orchestrator support AWS Transit Gateway? AWS Transit Gateway helps customers to interconnect virtual private clouds and their on-premises networks to a single gateway. Users only need to create and manage a single connection from the central gateway to each Amazon VPC, on-premises data center, or remote office across your network. It basically acts as a hub that controls how traffic is routed among all the connected networks which act like spokes. Aviatrix Orchestrator adds an automation layer to AWS Transit Gateway that allows users to provision and implement route domains securely and accurately. Users can automatically configure and propagate segmentation policies and leverage built-in troubleshooting and visualization tools for monitoring the entire environment. Some of the advantages of combining Aviatrix Orchestrator and AWS Transit Gateway include: Ensuring your AWS network follows virtual private cloud  segmentation best practices Limiting lateral movement in the event of a security breach Reducing the impact of human error by removing the need for potentially tedious manual configuration. Minimizing the blast radius that can result from misconfigurations. Replacing a flat architecture with a transit architecture Aviatrix Orchestrator is now available as an optional feature of the Aviatrix AVX Controller. New customers can launch the Aviatrix Secure Networking Platform AMI from AWS Marketplace to get access to this functionality. The existing customers can upgrade to the latest version of AVX software to use this feature. For more detail, visit the Aviatrix website. cstar: Spotify’s Cassandra orchestration tool is now open source! Amazon re:Invent announces Amazon DynamoDB Transactions, CloudWatch Logs Insights and cloud security conference, Amazon re:Inforce 2019 AWS re:Invent 2018: Amazon announces a variety of AWS IoT releases

A few days ago, the Linux Daemon (LXD) team announced the release of LXD 3.15. The major highlight of the release is the transition of LXD to the dqlite 1.0 branch, which will yield better performance and reliability to cluster users and standalone installations.  Linux Daemon (LXD) is a next-generation system container manager which uses Linux containers. It’s a free software, written in Go and developed under the Apache 2 license. LXD 3.15 explores new features including hardware VLAN and MAC filtering on SR-IOV, new storage-size option for lxd-p2c, Ceph FS storage backend for custom volumes and more. It also includes many major improvements including DHCP lease handling, cluster heartbeat handling, and bug fixes. What’s new in LXD 3.15? Hardware VLAN and MAC filtering on SR-IOV The security.mac_filtering and vlan properties are now available to SR-IOV devices. This will prevent MAC spoofing from the container as it will directly control the matching SR-IOV options on the virtual function. It will also perform hardware filtering at the VF level, in case of VLANs. New storage-size option for lxd-p2c A new --storage-size option has been added in LXD 3.15. When this option is used along with   --storage, it allows specifying the desired volume size to use for the container. Ceph FS storage backend for custom volumes Ceph FS is used as a storage driver for LXD and its support is limited to custom storage volumes. Its support includes size restrictions and native snapshot when the server, server configuration, and client kernel support those features. Ceph FS also allows attaching the same custom volume to multiple containers at the same time, even if they’re located on different hosts. IPv4 and IPv6 filtering IPv4 and IPv6 filtering (spoof protection) enable multiple containers to share the same underlying bridge, without worrying about spoofing the address of other containers, hijacking traffic or causing connectivity issues. Read Also: Internet governance project (IGP) survey on IPV6 adoption, initial reports Major improvements in LXD 3.15 Switch to dqlite 1.0 After a year of running all the LXD servers on the original implementation of distributed sqlite database, LXD 3.15 has finally switched to its 1.0 branch. This transition reduces the number of external dependencies, CPU usage and memory usage for the database. It also makes it easier to debug issues and integrate better with more complex database operations when running clusters.  Reworked DHCP lease handling In the previous versions, LXD’s handling of DHCP was pretty limited. With LXD 3.15, LXD will itself be able to issue DHCP requests to the dnsmasq server based on what’s currently in the DHCP lease table. This allows the user to manually release a lease when a container’s configuration is altered or a container is deleted, all without ever needing to restart dnsmasq. Reworked cluster heartbeat handling With LXD 3.15, the internal heartbeat (the list of database nodes) extends to include the most recent version information from the cluster as well as the status of all cluster members. This means that only the cluster leader will have to retrieve the data and the remaining members will get a consistent view of everything within 10s. Some of the Bug fixes in LXD 3.15 Linker flags have been updated. Path to the host’s communication socket has been fixed: doc/devlxd Basic install instructions have been added: doc/README Translations from weblate has been updated: i18n Unused arg from setNetworkRoutes has been removed: lxd/containers Unit tests have been updated: lxd/db Developers are happy with the new features and improvements included in LXD 3.15. A user on Reddit says, “The IPv4 and IPv6 spoof protection filters is going to make a few people very happy. As well as ceph FS support as RBD doesn't like sharing volumes with multiple host.” Some users were comparing LXD with Docker, where mostly all preferred the former over the latter. A Redditor gave a detailed comparison of the two platforms. The comment read, “The high-level difference is that Docker is for "application containers" and LXD is for "system containers". For Docker that means things like, say, your application process being PID 1, and generally being forced to do things the "Docker way".  “LXD, on the other hand, provides flexibility to use containers the way you want to. This means containers end up being closer to your development environment, e.g. by using systemd if you want it; they can be ephemeral like Docker, but only if you want to”, the user further added.  “So, LXD provides containers that are closer in feel to a regular installation or VM, but with the performance benefit of containers. You can even use LXD containers as Docker hosts, which is what I often do.” For the complete list of updates, head over to the LXD 3.15 release notes. LXD 3.11 releases with configurable snapshot expiry, progress reporting, and more LXD 3.8 released with automated container snapshots, ZFS compression support and more! Debian 10 codenamed ‘buster’ released, along with Debian GNU/Hurd 2019 as a port

Yesterday, Travis CI announced that its service will now be available on Windows. Travis CI is a distributed Continuous Integration service used to test and deploy projects hosted on GitHub. This is an early release and they plan to release a stable version in Q2 next year. With this update, teams can run their tests on Linux, Mac, and Windows--all in the same build. At present, users can use Windows with open source and private projects on either travis-ci.org or travis-ci.com. Travis CI plans to bring this to enterprise soon. The company says, “this is our very first full approach to Windows-support, so the tooling is light.” Laurie Voss, Chief Operating Officer, npm, Inc says, “Adding Windows support to Travis CI will provide a more stable development experience for a huge segment of the JavaScript community—32% of projects in the npm Registry use Travis CI. We look forward to continuing to work with Travis CI to reduce developer friction and empower over 10 million developers worldwide to build amazing things.” Travis Windows CI environment Windows Build Environment for Travis CI launches with support for Node.js, Rust, and Bash languages. Travis Windows CI will run a git bash shell, to maintain consistency with our other bash-based environments. This will also allow users to shell out to PowerShell as needed. In addition to this, Docker is also made available for Windows builds. Travis CI uses Chocolatey as a package manager and also has a pre-installed Visual Studio 2017 Build Tools. The Windows build environment is currently based on Windows Server 1803 for containers running Windows Server 2016 as the OS version. Travis CI in their blog post mention that they are hosting their Windows virtual machines in Google Compute Engine. Following which, they have seen some variations in their boot times. However, they plan to improve this alongside their other infrastructure-related work. The company expects to release Windows Build Environments for Enterprise before the release of the stable version. To know more about Travis CI on Windows in detail, visit their official Travis CI blog. Creating a Continuous Integration commit pipeline using Docker [Tutorial] How to master Continuous Integration: Tools and Strategies Why Agile, DevOps and Continuous Integration are here to stay: Interview with Nikhil Pathania, DevOps practitioner

DockerCon 2018 is around the corner and is taking place at the Moscone Center in San Francisco next week from 12th -15th June. More than 6,000 developers, architects, system admins, and other IT professionals are expected to get their hands on the latest enhancements in the container ecosystem. DockerCon is where people from the Docker community come together to learn, share, and collaborate. Here, you will find attendees from beginners, to intermediate and advanced experts who are interested in learning something new and enhancing their skill set. So, if you are interested in learning the modern ways of working with Docker then, this is your perfect chance. Here, you will have 2 full days of training, over a 100 session and hands-on labs, free workshops and more that will be brought to the table by different individuals. If you haven’t yet scheduled your DockerCon Agenda, here is the DockerCon Agenda Builder that will help you browse and search the sessions you are looking forward to in DockerCon 2018. With that being said, here are some interesting sessions you should not miss in your trip to DockerCon 2018. Automated Hardware Testing Using Docker for Space We already know how hard it is to cope up with space but that is not keeping Docker from thinking beyond web content. Space software development is difficult as they run on highly constrained embedded hardware. But Docker and its DevOps mentality helped DART create a scalable and rapidly deployable test infrastructure, in NASA’s mission to hit an asteroid at 6 km/s. This presentation will be all about how Docker can be used for both embedded development environment and scalable test environment. You will also learn about how Docker has evolved testing from a human-based testing to an automated one. Lastly, this presentation will summarize the do’s and don'ts of automated hardware testing, how you can play a key role in making a difference and what Docker wishes to achieve in the near future. Democratizing Machine Learning on Kubernetes One of the biggest challenges Docker is facing today is understanding how to build a platform that runs common open-source ML libraries such as Tensorflow. This session will be all about deploying distributed Tensorflow training cluster with GPU scheduling Kubernetes. This session will also teach you about the functioning of distributed training, its various options and which options to choose when. Lastly, this session will cover best practices on using distributed Tensorflow on top of Kubernetes. In the end, you will be provided with a public Github repository of the entire work presented in this session. Serverless Panel (Gloo function gateway) DockerCon 2018 is entirely based on your journey to containerization, where you will learn about modernizing traditional applications, adding microservices, and then serverless environments. One of the interesting development areas in 2018 is Gloo which is designed for microservice, monolithic, and serverless applications. It is a high-performance, plugin-extendable, platform-agnostic function gateway that enables the enterprise application developer to modernize a traditional application. Gloo containerizes a traditional application and uses microservices to add functions to it. Developers can then leverage orchestrated and routed portable serverless frameworks on top of Docker EE, or AWS Lambda to create hybrid cloud applications. Don’t Have A Meltdown! Practical Steps For Defending Your Apps With recent cybercrime events such as Meltdown and Spectre, security has become one of the major concerns for applications developers and operations teams. This session will demonstrate some best practices, configuration, and tools to effectively defend your container deployments from some common attacks. This session will be all about risks and preventive measures to be taken on authentication, injection, sensitive data, and more. All the events displayed in this session are inspired from highlights of OWASP Top 10 and other popular and massive attacks. By the end of this session, you will understand important security risks in your application and how you can go about mitigating them. Tips & Tricks of the Docker Captains This session is entirely focused on the tips and tricks for making the most out of Docker. These best practices will be from Docker Captains who will guide users in making common operations easier, addressing common misunderstandings, and avoiding common pitfalls. Topics covered in this session will revolve around build processes, security, orchestration, maintenance and more. This session will not only make new and intermediate user’s life easy with Docker but will also provide some new and valuable information to advanced users. DockerCon is considered as the number one container conference for IT professionals interested in learning and creating scalable solutions with innovative technologies. So, what are you waiting for? Start planning for DockerCon 2018 now and if you haven’t yet, you can register for DockerCon 2018 and get your container journey started. Related Links What’s new in Docker Enterprise Edition 2.0? Google Kubernetes Engine 1.10 is now generally available and ready for enterprise use Microsoft’s Azure Container Service (ACS) is now Azure Kubernetes Services (AKS)

We've teamed up with Humble Bundle once again to bring you a selection of some of our very best DevOps eBooks and videos for incredible prices. But more than that, we're also giving you the chance to support some really important charities while picking up the content you want. Go to Humble Bundle now. Get up to $1613 worth of DevOps eBooks and videos for $15! There are a diverse range of titles in the Humble Bundle. They cover all the essential components within the DevOps toolchain. This includes Automate It!, DevOps with Kubernetes, Deployment with Docker, and Ansible 2 for Beginners. With DevOps becoming an increasingly important to the way everyone builds software, this bundle of resources could prove vital for people even outside of sys admin type roles (apparently we're all sys admins now anyway). Pay at least $1 to pick up... Automate It! Effective DevOps with AWS DevOps for Web Developers [Video] Deployment with Docker Docker and Kubernetes for Java Developers Deploying and Running Docker Containers [Video] Linux Shell Scripting Solutions [Video] Three months subscription to Mapt pro for $30 ...Or pay at least $8 to get the titles above and... Ansible 2 for Beginners [Video] Practical Network Automation DevOps for Networking DevOps with Kubernetes Windows Server 2016 Automation with PowerShell Cookbook, Second Edition Learning Continuous Integration with Jenkins, Second Edition Getting Started with Kubernetes, Second Edition Mastering Puppet for Large Infrastructures [Video] ...Or pay at least $15 to get all of that and... Mastering Ansible, Second Edition Mastering DevOps [Video] Mastering Docker, Second Edition Getting Started with Docker [Video] Continuous Delivery with Docker and Jenkins Mastering Kubernetes Learning Kubernetes [Video] OpenStack Administration with Ansible 2, Second Edition Mastering Windows PowerShell 5 Administration [Video] Puppet 5 Beginner's Guide, Third Edition The Humble Bundle expires on 21 May 2018. So be quick and grab the DevOps eBooks and videos you want!