CloudPro

Announcing Terraform Google Provider 6.0.0 CloudPro #62: Kubernetes 1.31 Fine-grained SupplementalGroups control Quick Start Kubernetes Understand what Kubernetes is and why it's essential Learn the inner workings of Kubernetes architecture Get hands-on with deploying and managing applications Set up Kubernetes and containerize applications GET IT FOR $18.99 $12.99 ⭐Masterclass: Unlock the Full Potential of Kubernetes for Scalable Application Management Kubernetes pod and container restarting Better Kubernetes YAML Editing with (Neo)vim Monitoring kubernetes events with kubectl and Grafana Loki Practical Logging for PHP Applications with OpenTelemetry Using 1Password with External Secrets Operator in a GitOps way 🔍Secret Knowledge: Build your own SQS or Kafka with Postgres Revealing the Inner Structure of AWS Session Tokens An Opinionated Ramp Up Guide to AWS Pentesting Gang scheduling pods on Amazon EKS using AWS Batch multi-node processing jobs Application Availability Depends on Dependencies ⚡Techwave: Kubernetes 1.31: Fine-grained SupplementalGroups control Announcing Terraform Google Provider 6.0.0 New capabilities in VMware Private AI Foundation with NVIDIA GitLab Announces the General Availability of GitLab Duo Enterprise Grafana 11.2 release: new updates for data sources, visualizations, transformations, and more 🛠️HackHub: Best Tools for the Cloud PostgreSQL cloud native High Availability and more Kubernetes Operator to automate Helm, DaemonSet, StatefulSet & Deployment updates Runs and manages databases, message queues, etc on K8s Powerful workflow engine and end-to-end pipeline solutions implemented with native Kubernetes resources configure kubernetes objects on multiple clusters using jsonnet Cheers, Shreyans Singh Editor-in-Chief Mobile Banking Apps: Secure SDKs Aren’t Enough (Webinar) Is your mobile banking app truly secure? Join our webinar to learn why relying solely on protected SDKs leaves your app vulnerable. Discover real-world scenarios where emerging vulnerabilities can compromise your app despite using a protected SDK. We'll cover multi-layered protection strategies and practical solutions to guard against reverse engineering, tampering, and malware. Gain actionable insights on using obfuscation, data encryption, and real-time application self-protection (RASP) to safeguard your app. Equip yourself with practical solutions to ensure comprehensive app security and safeguard your business from financial and regulatory risks. REGISTER NOW Forward to a Friend ⭐MasterClass: Tutorials & Guides Kubernetes pod and container restarting In Kubernetes, a Pod is the smallest deployable unit, often containing one or more containers. When a container or pod needs to be restarted due to errors or updates, Kubernetes offers several methods to do so. For example, you can restart a Pod by deleting it, and Kubernetes will automatically recreate it if it’s part of a Deployment. Alternatively, you can restart a specific container within a Pod using commands like `kubectl exec` for more precise control. These features allow Kubernetes to maintain high availability and resilience in a cloud environment. Better Kubernetes YAML Editing with (Neo)vim Editing Kubernetes YAML files can be tricky, but using Neovim, a modern version of Vim, can make it much easier. Neovim is lightweight, highly customizable, and integrates well with your terminal, making it ideal for DevOps and platform engineers. By configuring Neovim specifically for YAML files, you can set up features like auto-indentation, syntax highlighting, folding, and autocompletion, all of which help reduce errors and improve efficiency. Monitoring kubernetes events with kubectl and Grafana Loki In Kubernetes, monitoring events is crucial for understanding the status and issues related to Pods, WorkerNodes, and other components. You can use `kubectl` to view these events directly, or you can enhance your monitoring setup by integrating Kubernetes events with Grafana Loki. By capturing events as logs using a tool like the `k8s-event-logger`, which listens to the Kubernetes API, you can store them in Loki, create metrics with RecordingRules, and visualize them in Grafana. Practical Logging for PHP Applications with OpenTelemetry Practical logging for PHP applications using OpenTelemetry involves instrumenting your PHP code to collect and correlate log data with other observability signals like traces and metrics. This approach is particularly useful in microservices-based architectures, where understanding the interactions between different services is crucial for maintaining system stability. By using OpenTelemetry, developers can standardize how telemetry data is collected and exported, reducing complexity. Using 1Password with External Secrets Operator in a GitOps way To manage secrets securely in a GitOps environment using Kubernetes, you can integrate 1Password with the External Secrets Operator. This setup allows you to automatically fetch and inject secrets stored in 1Password into your Kubernetes cluster. By using tools like ArgoCD, Helm, or FluxCD, you can deploy and manage this integration efficiently. The External Secrets Operator pulls secrets from 1Password via 1Password Connect, a proxy that ensures availability and reduces API requests. PACKT TITLES FOR YOU Buy now at $16.99 $10.99 Buy now at $39.99 $27.98 Buy now at $24.99 $16.99 🔍Secret Knowledge: Learning Resources Build your own SQS or Kafka with Postgres You can build your own version of SQS (Simple Queue Service) or Kafka using PostgreSQL by setting up tables and queries that mimic the functionality of these popular message queues and streams. For SQS, you create a table to store messages, with columns that help manage message visibility, delivery attempts, and order. You can then write queries to insert messages, retrieve them while respecting visibility timeouts, and delete them after processing. For Kafka, you expand this setup by storing messages persistently and keeping track of where each consumer group is in the message stream, allowing multiple consumers to process messages independently and in parallel, similar to Kafka's partitioning system. Revealing the Inner Structure of AWS Session Tokens By reverse engineering these tokens, the research team developed tools to analyze and modify them programmatically. This allowed them to uncover previously unknown details about AWS's cryptography and authentication protocols. Their findings showed that while AWS's security measures are robust, understanding the structure of these tokens can help defenders better protect against potential attacks. Additionally, the research raises questions about the privacy and integrity of these tokens. An Opinionated Ramp Up Guide to AWS Pentesting) Lizzie Moratti's "Opinionated Ramp Up Guide to AWS Pentesting" offers a detailed roadmap for becoming proficient in AWS pentesting, emphasizing practical experience over certifications. The guide is tailored for those with a foundational understanding of networking and security, and it stresses the importance of broad knowledge before delving into deeper cloud-specific skills. The guide also touches on industry pitfalls, such as reliance on automated tools and the challenges of cloud pentesting in a fast-evolving environment. Gang scheduling pods on Amazon EKS using AWS Batch multi-node processing jobs AWS Batch now supports multi-node parallel (MNP) jobs for Amazon EKS, allowing you to gang schedule pods across multiple nodes for tasks that require extensive computation, like machine learning or weather forecasting. Previously, MNP jobs were only available on Amazon ECS. With this update, you can use AWS Batch on EKS to run distributed processing jobs, such as those with Dask, a Python library for parallel computing. The setup involves defining job configurations that include a main node running the scheduler and worker nodes executing the tasks. This approach ensures efficient communication and scaling across nodes, streamlining complex computations in a managed environment. Application Availability Depends on Dependencies Modern applications depend on various services and components, meaning their reliability is tightly linked to the uptime of these dependencies. For example, if an application like Tekata.io needs to maintain 99.9% uptime, but it relies on several services with only 99.9% uptime each, the combined effect could reduce Tekata.io’s overall availability. To hit the desired uptime, dependencies need to have even higher availability. The formula \( A = U^N \) shows that if your application’s target uptime is 99.9% and it has 7 dependencies, each dependency must have an uptime of 99.99% to meet that target. ⚡TechWave: Cloud News & Analysis Kubernetes 1.31: Fine-grained SupplementalGroups control In Kubernetes 1.31, a new feature called `supplementalGroupsPolicy` was introduced to give better control over how supplementary group IDs are handled in Pods. Previously, Kubernetes automatically included group memberships defined in the container’s `/etc/group` file, which could lead to unexpected group IDs being applied and potentially cause security or access issues. With this update, you can now specify a `Strict` policy that only includes the group IDs explicitly set in the Pod's manifest, excluding any additional groups defined in the container image. Announcing Terraform Google Provider 6.0.0 The Terraform Google Provider 6.0.0 introduces several enhancements for better management of Google Cloud resources. Key updates include the option to opt-out of a default label ("goog-terraform-provisioned") that identifies Terraform-managed resources, improved protection against accidental resource deletion with new deletion protection fields, and increased flexibility with longer name prefixes for resources. New capabilities in VMware Private AI Foundation with NVIDIA Key updates in VMware Private AI include a Model Store for secure LLM management, a streamlined deployment process, and new NVIDIA capabilities like NIM Agent Blueprints for custom AI workflows. Future updates will include better GPU management, advanced data indexing and retrieval services, and tools for building AI agents. GitLab Announces the General Availability of GitLab Duo Enterprise GitLab has launched GitLab Duo Enterprise, an AI-powered add-on designed to enhance the software development lifecycle for DevSecOps teams. Priced at $39 per user per month, this tool integrates advanced AI features to improve code generation, security vulnerability detection, and team collaboration. It builds on the capabilities of GitLab Duo Pro by adding enterprise-focused tools like vulnerability resolution, root cause analysis, and AI impact dashboards. Grafana 11.2 release: new updates for data sources, visualizations, transformations, and more Notable additions include support for new data sources like Yugabyte and Amazon Managed Service for Prometheus, updates to visualizations such as standardized tooltips and pagination for state timelines, and improvements in transformations like data transposing and enhanced template variable support. The release also includes better alerting features, integration improvements for OAuth and SAML providers, and a migration assistant for easier transition to Grafana Cloud. 🛠️HackHub: Best Tools for Cloud sorintlab/stolon Stolon is a cloud-native tool designed to manage PostgreSQL databases with high availability, making it suitable for deployment in various environments including Kubernetes and traditional infrastructures. It leverages PostgreSQL's streaming replication and integrates with cluster stores like etcd, Consul, or Kubernetes for leader election and data storage. keel-hq/keel Keel is a lightweight tool for automating updates to Kubernetes deployments without needing complex command-line interfaces or APIs. It integrates directly with Kubernetes and Helm, using labels and annotations to manage updates based on semantic versioning policies. apecloud/kubeblocks KubeBlocks is an open-source tool designed to simplify the management of multiple database types on Kubernetes using a unified set of APIs. Instead of dealing with different operators for each database, KubeBlocks provides a single control plane to manage various databases such as PostgreSQL, Redis, and Kafka. It offers a standardized approach to database lifecycle management, day-2 operations, and observability, with support for backup, recovery, and monitoring. caicloud/cyclone Cyclone is a workflow engine built for Kubernetes that manages end-to-end pipelines without requiring extra dependencies. It operates across various Kubernetes environments, including public, private, and hybrid clouds. Cyclone offers features like DAG graph scheduling, flexible parameterization, and integration with external systems. It supports triggers, multi-cluster execution, multi-tenancy, and automatic resource cleanup. splunk/qbec Qbec is a CLI tool designed for managing Kubernetes objects across multiple clusters or namespaces using jsonnet, a data-templating language. It simplifies Kubernetes configuration management by allowing users to define and deploy objects in various environments efficiently. Qbec is similar to tools like kubecfg and ksonnet. 📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us. If you have any comments or feedback, just reply back to this email. Thanks for reading and have a great day! *{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}.reverse{display:table;width: 100%;

Github Copilot Autofix: Secure code 3x faster CloudPro #61: How Figma Migrated onto K8s in Less Than 12 months ⭐Masterclass: From Docker Compose to Kubernetes Manifests A hard look at GuardDuty shortcomings Streamlining Keycloak in Kubernetes The hater’s guide to Kubernetes A skeptic's first contact with Kubernetes 🔍Secret Knowledge: Enhancing Bitnami Helm Charts Security Cloudflare adopted OpenTelemetry for logging pipeline Josh Grose on LinkedIn: I spent the last 3 yrs outside of observability Did you know the CNCF has an actual cookbook? Not metaphorically! Unfashionably secure: why we use isolated VMs ⚡Techwave: How Figma Migrated onto K8s in Less Than 12 months Github Copilot Autofix: Secure code 3x faster New Kubernetes CPUManager Static Policy: Distribute CPUs Across Cores Announcing mandatory multi-factor authentication for Azure sign-in GitHub scales on demand with Azure Functions 🛠️HackHub: Best Tools for the Cloud Web tool for database management The devs are over here at devzat, chat over SSH! CloudFormation_To_Terraform Debugging tool for Kubernetes which tests and displays connectivity between nodes in the cluster. Kubernetes network solution Cheers, Shreyans Singh Editor-in-Chief Forward to a Friend ⭐MasterClass: Tutorials & Guides From Docker Compose to Kubernetes Manifests This blog post provides a beginner-friendly guide for developers transitioning from Docker Compose to Kubernetes manifests, using Minikube for local Kubernetes development. It walks through setting up Minikube, deploying a sample application using Docker Compose, converting Docker Compose files into Kubernetes manifests with Kompose, and finally deploying the application on Kubernetes. The guide emphasizes practical steps, like generating and applying Kubernetes manifests, and validating deployments using the Minikube dashboard. A hard look at GuardDuty shortcomings AWS GuardDuty, while a cornerstone in cloud threat detection, isn't without its flaws. It offers good coverage and deep integration with AWS services, but its limitations in service support, detection latency, and cost can leave gaps in your security posture. Adversarial simulations and benchmarks reveal that GuardDuty can miss critical threats, and its detection can be slow, especially for high-impact, low-volume attacks like S3 ransomware. Streamlining Keycloak in Kubernetes In this blog post, the author, a DevOps Engineer at Tikal, shares how they automated the deployment and management of Keycloak, an open-source identity and access management solution, within a Kubernetes environment. By leveraging Kubernetes’ native capabilities, Helm, and Python, they streamlined the complex configuration process, which typically requires extensive manual adjustments. This approach not only ensures consistency and reduces manual efforts but also enables scalable and repeatable deployments. The hater’s guide to Kubernetes Kubernetes often gets a bad rap for being overly complex, especially for startups with small teams. Critics argue it’s over-engineering for tasks that don't need such a heavyweight solution. The key to avoiding its complexity is to use only the necessary features and ignore the rest. While Kubernetes isn’t for everyone, especially for those needing quick, ephemeral workloads, it's a solid choice if you need the robustness it offers and are careful in its application. A skeptic's first contact with Kubernetes The author’s first real exploration of Kubernetes revealed its core concepts like control loops, services, and workload management, which actually simplify and automate many tasks traditionally done manually. Kubernetes uses controllers to ensure that workloads meet desired states, services to manage network traffic efficiently, and storage management to handle data persistence across pods. While the system has some quirks and limitations, its approach to automating and scaling workloads has proven to be a valuable evolution in managing modern infrastructure. Quick Start Kubernetes The course prepares you to leverage Kubernetes for continuous development and deployment. Whether you're scaling applications to meet demand or ensuring seamless updates with minimal downtime, you'll be equipped with the skills necessary for efficient and effective Kubernetes management. This course is your gateway to becoming proficient in one of the most essential tools in the DevOps toolkit. 🔍Secret Knowledge: Learning Resources Related Titles Enhancing Bitnami Helm Charts Security Bitnami enhanced the security of its Helm charts using Kubescape, an open-source Kubernetes security tool that identifies misconfigurations by comparing configurations to industry best practices. By integrating Kubescape into their build pipelines, Bitnami made significant improvements such as eliminating group root dependencies, configuring immutable filesystems, and reducing misconfigured resources. Cloudflare adopted OpenTelemetry for logging pipeline Cloudflare recently transitioned its logging pipeline from syslog-ng to OpenTelemetry Collector to enhance performance, maintainability, and telemetry insights. This move allowed the team to leverage Go, a language more familiar to their engineers, and integrate better observability through Prometheus metrics. Despite challenges like minimizing downtime during the switch and ensuring compatibility with existing infrastructure, the migration has opened up opportunities for further improvements, such as better log sampling and migration to the OpenTelemetry Protocol (OTLP). Josh Grose on LinkedIn: I spent the last 3 yrs outside of observability Josh Grose (ex-Principal PM, Splunk), after three years away from the observability space, was surprised to find that despite companies spending around 30% of their cloud budgets on monitoring, reliability hasn't improved significantly. He observed that even when Service Level Agreements (SLAs) are met, it often comes at the cost of developer productivity and experience. Engineering leaders are frustrated with the high costs and limited improvements in key metrics like Mean Time to Recovery (MTTR) and development speed, leading to the perception that observability has become an expensive and ineffective necessity. Did you know the CNCF has an actual cookbook? Not metaphorically! The "Cloud Native Community Cookbook" is a unique collection of recipes put together by the CNCF and Equinix Metal, born out of the increased time people spent at home during the COVID-19 pandemic. Instead of focusing on cloud technologies, this cookbook brings together food recipes shared by members of the Cloud Native community, originally exchanged in Equinix Metal's Slack channel. Unfashionably secure: why we use isolated VMs While modern cloud architectures often favor shared, multi-tenant environments for efficiency and scalability, Thinkst Canary opts for a less trendy but highly secure approach by using isolated virtual machines (VMs) for each customer. This choice prioritizes security by ensuring that each customer's data and services are completely separated, reducing the risk of cross-customer data breaches. Although this method comes with higher operational costs and complexity, it provides a stronger security boundary, making it easier to manage risks and sleep better at night. ⚡TechWave: Cloud News & Analysis How Figma Migrated onto K8s in Less Than 12 months Figma completed its migration to Kubernetes in under a year by meticulously planning and executing a well-scoped transition. Initially running services on AWS's ECS, Figma faced limitations such as complex stateful workloads and limited auto-scaling. The decision to move to Kubernetes (EKS) was driven by its broader functionality, including support for StatefulSets, Helm charts, and advanced scaling options from the CNCF ecosystem. By Q1 2024, Figma had migrated most core services with minimal impact on users, resulting in enhanced reliability, reduced costs, and a more flexible compute platform. Github Copilot Autofix: Secure code 3x faster Copilot Autofix, now available in GitHub Advanced Security, is an AI-powered tool designed to help developers fix code vulnerabilities more than three times faster than manual methods. It analyzes vulnerabilities, explains their significance, and offers code suggestions for quick remediation. This accelerates the fixing process for both new vulnerabilities and existing security debt, significantly reducing the time and effort required for secure coding. Copilot Autofix is included by default for GHAS customers and also available for open source projects starting in September. New Kubernetes CPUManager Static Policy: Distribute CPUs Across Cores Kubernetes v1.31 introduces a new alpha feature called "distribute-cpus-across-cores" for the CPUManager's static policy. This option aims to enhance performance by spreading CPUs more evenly across physical cores, rather than clustering them on fewer cores. This reduces contention and resource sharing between CPUs on the same core, which can boost performance for CPU-intensive applications. To use this feature, users need to adjust their Kubernetes configuration to enable it. Currently, it cannot be combined with other CPUManager options, but future updates will address this limitation. Announcing mandatory multi-factor authentication for Azure sign-in Microsoft is making multi-factor authentication (MFA) mandatory for all Azure sign-ins to enhance security and protect against cyberattacks. Starting in the latter half of 2024, Azure users will need to use MFA to access the Azure portal and admin centers, with broader enforcement for other Azure tools like CLI and PowerShell set for early 2025. MFA, which adds an extra layer of security by requiring more than just a password, is shown to block over 99% of account compromises. GitHub scales on demand with Azure Functions GitHub faced scalability issues with its internal data pipeline, which struggled to handle the massive amount of data it collects daily. To address this, GitHub partnered with Microsoft to use Azure Functions' new Flex Consumption plan, which allows serverless functions to scale dynamically based on demand. This solution has enabled GitHub to efficiently process up to 1.6 million events per second, addressing their growth challenges and improving performance with minimal overhead. 🛠️HackHub: Best Tools for Cloud commandprompt/pgmanage PgManage is a modern graphical database client for PostgreSQL, focusing on management features and built on the now-dormant OmniDB project. quackduck/devzat Devzat is a chat service accessible via SSH that replaces the traditional shell prompt with a chat interface, allowing you to connect from any device with SSH capabilities. aperswal/CloudFormation_To_Terraform The CloudFormation to Terraform Converter is a tool that automates the migration of AWS CloudFormation templates to Terraform configuration files. bloomberg/goldpinger Goldpinger monitors Kubernetes networking by making calls between its instances and providing Prometheus metrics for visualization and alerts. ZTE/Knitter Knitter is a Kubernetes CNI plugin that supports multiple network interfaces for pods, allowing custom network configurations across various cloud environments. Buy now at $16.99 $10.99 Buy now at $39.99 $27.98 Buy now at $24.99 $16.99 📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us. If you have any comments or feedback, just reply back to this email. Thanks for reading and have a great day! *{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{line-height:0;font-size:75%} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}