Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

DataPro

62 Articles
Merlyn from Packt
24 Jul 2025
Save for later

Amazon’s Mitra – Tabular Foundation Model, Qwen3-Coder-480B-A35B-Instruct, NVIDIA’s Cosmos DiffusionRenderer, DeepSeek R1 on Vertex AI

Merlyn from Packt
24 Jul 2025
Torchvista, AWS Data Processing MCP Server, Amazon Q + DLC MCP, Streamlit + MCP, ChatGPT AgentBecome an AI Generalist that makes $100K (in 16 hours)Still don’t use AI to automate your work & make big $$? You’re way behind in the AI race. But worry not:Join the World’s First 16-Hour LIVE AI Upskilling Sprint for professionals, founders, consultants & business owners like you. Register Now (Only 500 free seats)Date: Saturday and Sunday, 10 AM - 7 PM.Rated 4.9/10 by global learners – this will truly make you an AI Generalist that can build, solve & work on anything with AI.In just 16 hours & 5 sessions, you will:✅ Learn the basics of LLMs and how they work.✅ Master prompt engineering for precise AI outputs.✅ Build custom GPT bots and AI agents that save you 20+ hours weekly.✅ Create high-quality images and videos for content, marketing, and branding.✅ Automate tasks and turn your AI skills into a profitable career or business.All by global experts from companies like Amazon, Microsoft, SamurAI and more. And it’s ALL. FOR. FREE. 🤯 🚀$5100+ worth of AI tools across 2 days — Day 1: 3000+ Prompt Bible, Day 2: Roadmap to make $10K/month with AI, additional bonus: Your Personal AI Toolkit Builder.Register Now (Only 500 free seats)SponsoredSubscribe|Submit a tip|Advertise with usWelcome to DataPro #143: From Bits to Brains - The Tools Driving the Next Wave of Intelligent Systems 🧠📡What if your database could talk back with charts, or your containers built themselves when you spoke? What if your AI agent could say “I don’t know” and actually mean it?This week, we dive into a new breed of tools designed not just to build smarter systems, but to understand, reason, and scale them. These aren’t just marginal upgrades, they’re foundational shifts in how we build and interact with AI.Start with Mitra: Amazon’s tabular foundation model that ditches real-world data for synthetic priors (think causal graphs + tree ensembles) and still manages SOTA across tabular benchmarks via in-context learning.Then check out Qwen3-Coder-480B-A35B-Instruct, a Claude-class code model with 256K native context and 1M with Yarn, engineered for repository-scale agentic reasoning.Want BI that speaks SQL and your language? Wren AI is your GenBI agent, natural language in, SQL and insights out, thanks to a semantic layer, LLM integrations, and plug-and-play APIs.Visual domains aren’t left out. Cosmos DiffusionRenderer from NVIDIA reinvents video re-lighting with neural inverse rendering, 70GB models, and GPU-optimized pipelines for stunning realism.If you’re building with agents, 7 MCP Best Practices are a must-read, from schema validation to Dockerized deployments to performance tuning at scale.Meanwhile, ChatGPT Agent blurs the line between reasoning and doing, browsing, coding, and summarizing, all on its own virtual machine.But let’s not forget the human side. How Not to Mislead with Your Data is a masterclass on spotting narrative bias in data storytelling, and the ethical stakes behind our charts.And yes, Cloud SQL meets Vertex AI now means vector search and Gemini are just SQL calls away. You can embed, search, and analyze, all inside your relational DB.In the wild, Streamlit + MCP brings it all together in a sleek client interface that lets users query DeepWiki or HuggingFace-backed agents via natural language, no frontend dev required.AWS Data Processing MCP Server takes that to an enterprise level, streamlining schema discovery, query generation, and job monitoring across Glue, Athena, and EMR, all via natural language.Then, go deep with Amazon Q + DLC MCP: a system that automates PyTorch/TensorFlow container orchestration with a single prompt. Think: “Deploy PyTorch for multi-node training”, and it just happens.Finally, DeepSeek R1 on Vertex AI means no GPUs needed, just an API call. Run it on-demand, serverless, pay-as-you-go, no infrastructure stress.Still thinking of attention heads asdot products? Transformers as Addition Machines reframes attention with mechanistic interpretation, revealing layer-by-layer logic circuits.Or maybe you prefer pictures, Torchvista lets you trace PyTorch forward passes as interactive graphs inside your notebook, a dream for debugging or demystifying hidden layers.Semantic communication is making machines communicate with meaning, not bits. It’s the end of false alarms and overfitting to known categories, and it's all because of the knowledge graphs that reason over context and uncertainty.And if you’re ready to start building today, Google Cloud’s top 25 guides are a treasure trove: from RAG, RLHF, and agent orchestration to CI/CD pipelines and multi-agent chat apps, code included, no excuses.We’re in the midst of a shift: From models that classify to systems that reason. From dashboards to agents. From pixels to meaning.This issue is your map. Dive in, experiment, build.Sponsored: Your data, built your way with Twilio Segment — a customer data platform designed to cut through the chaos, unify your stack, and free you to focus on innovation over integration. Learn more.Cheers,Merlyn ShelleyGrowth Lead, PacktTop Tools Driving New Research 🔧📊⏩ Mitra: Mixed synthetic priors for enhancing tabular foundation models. Amazon’s Mitra is a tabular foundation model (TFM) that uses in-context learning to generalize across tabular tasks without retraining. Pretrained on synthetic data from causal models and tree-based methods, rather than real-world data, Mitra achieves state-of-the-art results across benchmarks like TabRepo and TabArena. It’s open source via AutoGluon 1.4.⏩ Qwen/Qwen3-Coder-480B-A35B-Instruct · Qwen3-Coder-480B-A35B-Instruct is Qwen’s most advanced code model, delivering Claude Sonnet-level performance on agentic coding and browser-use tasks. It supports 256K token context (extendable to 1M), tool calling, and repository-scale understanding. Built with 480B parameters (35B active), it uses in-context prompting and excels at function-call reasoning, agent frameworks, and long-horizon completions.⏩ Wren AI is your GenBI Agent: Wren AI is a GenBI agent that lets you query databases in natural language to generate SQL, charts, and AI-driven insights instantly. It features a semantic layer for governed accuracy, integrates with top LLMs, supports embedding via API, and connects to major data sources. Fast setup, cloud and open-source options included.⏩ nv-tlabs/cosmos1-diffusion-renderer: Cosmos DiffusionRenderer is NVIDIA’s latest video diffusion framework for high-quality image and video de-lighting and re-lighting. Built on DiffusionRenderer and powered by Cosmos, it features neural inverse and forward rendering with significant improvements in realism and control. It supports GPU-efficient inference, 70GB models, and full relighting pipelines for both static images and dynamic videos.Topics Catching Fire in Data Circles 🔥💬⏩ 7 MCP Server Best Practices for Scalable AI Integrations in 2025: Model Context Protocol (MCP) servers are becoming essential for secure, scalable, and agentic AI integrations. This guide outlines 7 best practices, toolset design, proactive security, schema validation, local/remote testing, Docker packaging, performance tuning, and documentation, that reduce errors, boost developer adoption, and power industry-wide AI success across finance, healthcare, e-commerce, and more.⏩ ChatGPT Agent: Bridging Research and Action: ChatGPT Agent introduces a powerful leap in agentic AI: it can now think and act on your behalf using its own virtual computer, navigating websites, running code, analyzing data, and producing editable outputs like slides and spreadsheets. It integrates browsing, terminals, APIs, and tool access to complete complex real-world tasks autonomously.⏩ How Not to Mislead with Your Data-Driven Story? Data storytelling helps us understand the world, but it can also mislead. This piece explores how persuasive narratives, even with accurate data, can distort truth. It highlights narrative bias risks like selection, framing, and interpretation, and urges data professionals to balance emotional storytelling with clarity, ethics, and rigorous data literacy.⏩ Integrate your Cloud SQL for MySQL instance with Vertex AI and vector search: Google Cloud’s Cloud SQL for MySQL now supports vector embeddings and Vertex AI integration, empowering developers to run AI-powered search and analysis directly in SQL. You can generate, store, and search vector embeddings with native SQL functions, perform ANN search, and invoke Gemini or custom Vertex AI models to assess customer sentiment or predict behavior, all within your database.New Case Studies from the Tech Titans 🚀💡⏩ MCP Client Development with Streamlit: Build Your AI-Powered Web App. This tutorial walks you through building a Streamlit-based MCP client interface that connects to remote MCP servers like DeepWiki and HuggingFace. The client lets users input topics and receive AI-generated summaries or recommendations via OpenAI’s API. It covers setup, secure key handling, MCP tool integration, and UI design, enabling rapid, modular deployment of AI-powered web tools.⏩ Accelerating development with the AWS Data Processing MCP Server and Agent: The AWS Data Processing MCP Server simplifies complex analytics workflows by enabling AI-driven natural language interactions with services like AWS Glue, Athena, and EMR. Built on the Model Context Protocol (MCP), it abstracts multi-service orchestration, automating tasks like schema discovery, query generation, reporting, and monitoring. Developers can integrate it via Amazon Q CLI or Claude Desktop to streamline onboarding, accelerate insight generation, and enhance observability.⏩ Streamline deep learning environments with Amazon Q Developer and MCP: Amazon Q + the DLC MCP Server radically simplifies how AI/ML teams manage Deep Learning Containers. Instead of manually customizing, testing, and deploying DLCs for PyTorch or TensorFlow, developers can now use natural language via Amazon Q CLI to automate everything, from image selection to ECR deployment, distributed training, and environment troubleshooting. It turns container operations into secure, conversational workflows.⏩ Deepseek R1 is available for everyone in Vertex AI Model Garden: DeepSeek R1 is now available on Vertex AI’s Model-as-a-Service (MaaS) platform, enabling businesses to access this powerful open model without managing GPU infrastructure. With just a few clicks or API calls, teams can test and deploy DeepSeek via a serverless, pay-as-you-go model. Vertex AI handles security, scalability, and compliance, accelerating AI innovation with zero infrastructure overhead.Blog Pulse: What’s Moving Minds 🧠✨⏩ Transformers (and Attention) are Just Fancy Addition Machines: Mechanistic interpretation is a novel AI interpretability approach that goes beyond tools like SHAP and LIME by uncovering how neural networks compute, not just what features influence outputs. It traces how features are encoded and transformed across layers, especially in transformers. By reimagining multi-head attention as additive rather than concatenative, it enables circuit-level analysis of neuron behavior. This method reveals the internal logic of models, opening doors to deeper understanding, debugging, and trust in complex AI systems.⏩ Torchvista: Building an Interactive Pytorch Visualization Package for Notebooks. Torchvista is an open-source tool for interactively visualizing the forward pass of PyTorch models inside web-based notebooks like Colab or Jupyter. Unlike static tools, it offers zoomable, modular graph views, supports error-tolerant partial visualizations, and requires just a one-line trace_model() call. It traces tensor flows and module hierarchies during forward execution and renders them as interactive, nested graphs using JS libraries like D3 and Graphviz, making complex models understandable, debuggable, and more accessible for iterative development and exploration.⏩ From Rules to Relationships: How Machines Are Learning to Understand EachOther? Semantic communication shifts focus from transmitting raw bits to conveying meaning, crucial in modern, machine-heavy networks. Traditional SKB systems compress messages via fixed categories, but fail in unfamiliar scenarios. Knowledge graph-based semantic communication fixes this by modeling relationships between entities, enabling contextual reasoning. This allows systems to intelligently handle edge cases (e.g., maintenance workers during off-hours) by inferring intent and suggesting verification over false alarms. Though graph systems require more compute and expertise, they vastly improve real-world accuracy, adaptability, and decision-making in noisy, dynamic environments.⏩ 25 top how-to guides for Google Cloud: The best way to learn AI is to build it, and Google Cloud now offers a curated collection of 25+ hands-on how-to guides to help you do just that. From deploying large models like Llama 3 and DeepSeek on high-performance infrastructure, to creating advanced gen AI apps, fine-tuning with RAG and RLHF, and integrating agents with real-world systems, this living resource accelerates your AI journey. Each guide includes code, tools, and best practices, ready to help you build smarter, faster, and at scale.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 1
  • 0

Merlyn from Packt
16 Jul 2025
Save for later

Amazon EKS now scales to 100K nodes, AutoKeras/Keras Tuner, Streamlit apps to AWS, Strands Agents 1.0

Merlyn from Packt
16 Jul 2025
NVIDIA’s Audio Flamingo 3, GoogleSQL’s new pipe syntax, MetaStone-S1, Fractional ReasoningAn Exclusive Look into Next Gen BI – Live WebinarDashboards alone aren’t cutting it. The market’s moving toward something new: data apps, live collaboration, and AI that works the way teams actually work.See what's driving the rise of Next Gen BI, how Sigma earned a top debut on the Gartner Magic Quadrant, and what’s next for our roadmap.Secure Your SpotSponsoredSubscribe|Submit a tip|Advertise with usWelcome to DataPro 142: Tools Driving Tomorrow’s Thinking 🔬📈In this edition, we spotlight the breakthrough tools, patterns, and practices that are reshaping research and production in AI and data science.From NVIDIA’s Audio Flamingo 3 pushing the frontier of multimodal reasoning, to Fractional Reasoning’s elegant solution to adaptive LLM compute, and MetaStone-S1’s bold performance claims, this week’s releases are not just incremental; they’re foundational. Meanwhile, Kiro is redefining the dev experience, merging agentic coding with production-readiness from day one.On the systems front, Amazon EKS now scales to 100K nodes, opening the door to AGI-class workloads. And GoogleSQL’s new pipe syntax is winning hearts in the SQL community for its clarity and composability. If you’ve ever loathed nested subqueries, this is your moment.For those making decisions about tooling, don’t miss our link on Foundation vs. Custom Models, a smart, grounded guide for teams navigating performance vs. control. Also featured: Amazon SageMaker’s new unified catalog, practical AutoML with AutoKeras/Keras Tuner, and a no-fuss walkthrough of deploying Streamlit apps to AWS.Lastly, we dive into deeper reflections: Strands Agents 1.0 brings multi-agent orchestration into the real world, and standout articles explore paradox pitfalls in metrics, and how data’s 40-year evolution is shaping AI’s next wave.Let’s get into it. ⬇️Cheers,Merlyn ShelleyGrowth Lead, PacktTop Tools Driving New Research 🔧📊🔵 nvidia/audio-flamingo-3 · Audio Flamingo 3 (AF3) is an open Large Audio-Language Model (LALM) by NVIDIA for research use, capable of reasoning across speech, sound, and music. It supports long audio inputs, multi-turn voice dialogue, and chain-of-thought reasoning, achieving state-of-the-art results on 20+ tasks through unified audio representation and extensive dataset training.🔵 Fractional Reasoning via Latent Steering Vectors Improves Inference Time Compute: Fractional Reasoning introduces a model-agnostic, training-free method to dynamically adjust LLM reasoning depth at inference. By scaling latent steering vectors, it tailors compute per input complexity, boosting accuracy and efficiency. Compatible with Best-of-N, majority vote, and self-reflection, it outperforms fixed prompts across GSM8K, MATH500, and GPQA benchmarks.🔵 MetaStone-AI/MetaStone-S1: MetaStone-S1 is a 32B-parameter reflective generative model that rivals OpenAI-o3-mini on math, code, and Chinese reasoning. It combines Long-CoT Reinforcement and Process Reward Learning for efficient, high-quality inference. MetaStone-S1 achieves deep reasoning while reducing policy model costs by 99%, enabling fast, accurate outputs across multiple benchmarks.🔵 Introducing Kiro: Kiro is an agentic IDE that turns AI prototypes into production-grade apps using spec-driven development. It auto-generates requirements, design docs, and implementation tasks, and uses hooks for event-based automation. With built-in test coverage, design clarity, and consistency checks, Kiro helps developers ship reliable software faster and with greater confidence.Topics Catching Fire in Data Circles 🔥💬🔵 Do You Really Need a Foundation Model? Not every use case needs a foundation model. This guide compares foundation and custom models across performance, cost, latency, and control. It offers a decision framework, practical examples, and hybrid strategies to help teams choose the right approach, balancing rapid prototyping with long-term scalability, privacy needs, and task-specific optimization.🔵 Automating Deep Learning: A Gentle Introduction to AutoKeras and Keras Tuner. This guide introduces AutoKeras and Keras Tuner, two AutoML tools that simplify deep learning. AutoKeras automates architecture and training, while Keras Tuner optimizes hyperparameters of custom models. Together, they streamline experimentation, reduce guesswork, and boost performance, ideal for tasks like image classification, tabular modeling, or rapid prototyping with minimal manual tuning.🔵 Amazon EKS enables ultra scale AI/ML workloads with support for 100K nodes per cluster: Amazon EKS now supports up to 100,000 nodes per cluster, enabling ultra-scale AI/ML workloads with 1.6M Trainium or 800K GPU instances. This breakthrough powers large model training, reduces operational costs, and preserves Kubernetes compatibility, paving the way for AGI-scale innovation through enhanced orchestration, resiliency, and open-source flexibility.🔵 Exploring pipe syntax real-world use cases: GoogleSQL's pipe syntax reimagines SQL with a linear, readable data flow using the |> operator. It simplifies complex queries, streamlines data pipelines, and improves log analysis clarity. By eliminating nested structures and enabling intuitive chaining, pipe syntax boosts productivity, maintainability, and accelerates insight generation across BigQuery and Cloud Logging workflows.New Case Studies from the Tech Titans 🚀💡🔵 How Metrics (and LLMs) Can Trick You: A Field Guide to Paradoxes. This article unpacks how paradoxes like Simpson’s, the Accuracy Paradox, and Goodhart’s Law mislead both data science and LLM evaluation. It shows how surface-level metrics can distort truth, urging practitioners to embrace contextual, nuanced measurement, especially in BI and Retrieval-Augmented Generation, where incentives, imbalance, and aggregation errors can derail decision-making.🔵 What Can the History of Data Tell Us About the Future of AI? This sweeping 40-year history of data explores how shifts in storage, architecture, and business models have shaped intelligent systems. By tracing personal, public, and enterprise data, from PCs to cloud to AI, the piece reveals how incentives, infrastructure, and data ownership will determine the trajectory of AI’s future.🔵 Streamline the path from data to insights with new Amazon SageMaker Catalog capabilities: Amazon SageMaker now streamlines analytics with new integrations: QuickSight for in-studio dashboarding, S3 Access Grants for secure unstructured data sharing, and automatic onboarding of Glue Data Catalog datasets. These updates unify structured and unstructured data, accelerating workflows from raw data to insights, governed, discoverable, and ready for ML and BI use.Blog Pulse: What’s Moving Minds 🧠✨🔵 Deploy a Streamlit App to AWS: This hands-on guide walks you through deploying a Streamlit app on AWS using Elastic Beanstalk. It covers preparing your code, switching from Postgres to S3 for data, configuring AWS infrastructure, and managing deployment steps. Ideal for developers needing scalable, secure alternatives to public cloud endpoints like Streamlit Community Cloud.🔵 Accuracy Is Dead: Calibration, Discrimination, and Other Metrics You Actually Need. This guide challenges accuracy as a primary evaluation metric, urging data scientists to adopt deeper, problem-specific tools. It explores advanced classification metrics like ROC-AUC, log loss, and Brier score, and regression metrics like R², RMSLE, and quantile loss, emphasizing calibration, uncertainty, and decision-readiness over surface-level model performance.🔵 Introducing Strands Agents 1.0: Production-Ready Multi-Agent Orchestration Made Simple: Strands Agents 1.0 is a production-ready SDK for building multi-agent AI systems. It introduces primitives like Agents-as-Tools, Swarms, Graphs, and A2A support for inter-agent communication. With session persistence, async performance, and flexible model integration, Strands simplifies orchestration, scaling from prototype to production for complex, collaborative, and distributed agentic workflows.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0

Merlyn from Packt
09 Jul 2025
Save for later

SmolLM3, Hugging Face’s small-but-mighty multilingual model with 128k-token context, MLarena, a diagnostic-rich, algorithm-agnostic toolkit

Merlyn from Packt
09 Jul 2025
Microsoft’s Copilot Chat goes open-source, Beyond Prompts: The Rise of Context EngineeringTogether with Growth School & Infinite UptimeJoin this 16 hour AI Learning Sprint to become an AI Genius (worth $895 but $0 today)The AI race is getting faster & dirtier day by day. Things we could never have imagined are happening.--Thousands of people are getting laid off everyday--People are building 1-person million dollar companies--Tech giants are fighting for AI talentMeta just poached OpenAI’s 4 top researchers …….So if you’re not learning AI today, you probably won't have a job in the next 6 months.That’s why, you need to join the 3-Day Free AI Mastermind by Outskill which comes with 16 hours of intensive training on AI frameworks, building with sessions, creating images and videos etc. that will make you an AI expert. Originally priced at $895, but the first 100 of you get in for completely FREE! Extended 4th of july SALE! 🎁📅FRI-SAT-SUN- Kick Off Call & Live Sessions🕜10AM EST to 7PM EST✅ trusted by 4M+ learnersIn the 5 sessions, you will:✅ Master prompt engineering to get the best out of AI.✅ Build custom GPT bots & AI agents for email management to save you 20+ hours weekly.✅ Create high-quality images and videos for PPTs, marketing, and branding.✅ Monetise your AI skills into a $10,000/mo business.All by global experts from companies like Amazon, Microsoft, SamurAI and more. And it’s ALL. FOR. FREE. 🤯 🚀Join now and get $5100+ in additional bonuses$5100+ worth of AI tools across 3 days — Day 1: 3000+ Prompt Bible, Day 2: Roadmap to make $10K/month with AI, Day 3: Your Personal AI Toolkit Builder.SponsoredSubscribe|Submit a tip|Advertise with usWelcome to DataPro #141 ~ Engineering Intelligence, Not Just ModelsIn this landmark edition, we go beyond algorithms and hyperparameters to explore how data science is evolving into a discipline of system design, orchestration, and reasoning. As GenAI shifts the boundaries of what’s possible, the conversation is no longer about what model to use, but how we structure intelligence itself.Our feature deep dive, “Beyond Prompts: The Rise of Context Engineering” byRahul Singh, Data Science Manager at Adobe,challenges the prompt-centric mindset and introduces Context Engineering as a foundational pillar for building scalable, intelligent agents. If you’re architecting the future of enterprise AI, this is essential reading.Also inside:– Build a fully autonomous multi-agent system with Python, OpenAI API, and PrimisAI Nexus– Explore SmolLM3, Hugging Face’s small-but-mighty multilingual model with 128k-token context– Microsoft’s Copilot Chat goes open-source, offering powerful AI pair programming to everyone– Google’s MCP Toolbox simplifies secure, schema-aware database access for AI agents– A technical teardown of Shazam’s algorithmic magic, from FFT to hash matching– How POSETs in Python provide better multi-criteria decisions than rankings– Launch smarter ML pipelines with MLarena, a diagnostic-rich, algorithm-agnostic toolkit– Unlock true concurrency with free-threaded Python 3.13 and StaticFrame for blazing-fast row opsWhether you're scaling models, building infrastructure, or shaping AI policy, this issue delivers insights for every data scientist at the frontier.✉️ Have tips or tools to share? Reply and contribute to our next edition.Cheers,Merlyn ShelleyGrowth Lead, PacktUnlock 99.97% Availability with PlantOS: Production Reliability, RedefinedPlantOS Manufacturing Intelligence is powering the next era of industrial performance — delivering 99.97% equipment availability and up to 2% energy savings per unit produced. From steel to cement, manufacturers worldwide are turning fragmented data into confident decisions across every layer of production — from parameter to plant to global scale.Experience Infinite Uptime NowSponsoredBeyond Prompts: The Rise of Context EngineeringWhy context engineering is the next frontier in building smarter, more reliable AI systems.Written by Rahul Singh, Data Science Manager @Adobe. Over my seven-plus-year career in data science, working on projects ranging from customer-value measurement to product analytics and personalization, one question has remained constant through it all:Do we have the right data, and can we trust it?With the rapid rise of Generative AI, that question hasn’t disappeared; it’s become even more urgent. As AI systems evolve from proof-of-concept assistive chatbots to autonomous agents capable of reasoning and acting, their success increasingly depends not on how complex or powerful they are, but on how well they understand the context in which they operate.In recent weeks, leaders like Tobi Lütke (CEO of Shopify), Andrej Karpathy (former Director of AI at Tesla), and others have spotlighted this shift. Lütke’s tweet was widely reshared, including by Karpathy, who elaborated on it further. He emphasized that context engineering is not about simple prompting, but about carefully curating, compressing, and sequencing the right mix of task instructions, examples, data, tools, and system states to guide intelligent behavior. This emerging discipline, still poorly understood in most organizations, is quickly becoming foundational to any serious application of generative AI.This growing attention tocontext engineeringsignals a broader shift underway in the AI landscape. For much of the past year,prompt engineeringdominated the conversation, shaping new job titles and driving a surge in hiring interest. But that momentum is tapering. A Microsoft survey across 31 countries recently ranked “Prompt Engineer” near the bottom of roles companies plan to hire(Source).Job search trends reflect the change as well: according to Indeed, prompt-related job searches have dropped from144 per milliontojust 20–30(Source).But this decline doesn’t signal the death of prompt engineering by any means. Instead, it reflects a field in transition. As use cases evolve from assistive to agentic AI, ones that can plan, reason, and act autonomously, the core challenge is no longer just about phrasing a good prompt. It’s about whether the model has the right information, at the right time, to reason and take meaningful action.This is where Context Engineering comes in!Suppose prompt engineering is about writing the recipe, carefully phrased, logically structured, and goal-directed. In that case,context engineeringis about stocking the pantry, prepping the key ingredients, and ensuring the model remembers what’s already been cooked. It’s the discipline of designing systems that feed the model relevant data, documentation, code, policies, and prior knowledge, not just once, but continuously and reliably.In enterprises, where critical knowledge is often proprietary and fragmented across various platforms, including SharePoint folders, Jira tickets, Wiki pages, Slack threads, Git Repositories, emails, and dozens of internal tools, the bottleneck for driving impact with AI is rarely the prompt. It’s the missing ingredients from the pantry, the right data, delivered at the right moment, in the right format. Even the most carefully crafted prompt will fall flat if the model lacks access to the organizational context that makes the request meaningful, relevant, and actionable.And as today’s LLMs evolve intoLarge Reasoning Models(LRM), and agentic systems begin performing real, business-critical tasks, context becomes the core differentiator. Models like OpenAI’s o3 and Anthropic’s Claude Opus 4 can handle hundreds of thousands of tokens in one go. But sheer capacity is not enough to guarantee success. What matters is selectively injecting the right slices of enterprise knowledge: source code, data schemas, metrics, KPIs, compliance rules, naming conventions, internal policies, and more.This orchestration of context is not just document retrieval; it’s evolving into a new systems layer. Instead of simply fetching files, these systems now organize and deliver the right information at the right step, sequencing knowledge, tracking intermediate decisions, and managing memory across interactions. In more advanced setups, supporting models handle planning, summarization, or memory compression behind the scenes, helping the primary model stay focused and efficient. These architectural shifts are making it possible for AI systems to reason more effectively over time and across tasks.Without this context layer, even the best models stall on incomplete or siloed inputs. With it, they can reason fluidly across tasks, maintain continuity, and deliver compounding value with every interaction.Case in point:This isn’t just theory. One standout example comes from McKinsey. Their internal GenAI tool,Lilli,is context engineering in action. The tool unifies over 40 knowledge repositories and 100,000+ documents into a single searchable graph. When a consultant poses a question, it retrieves the five to seven most relevant artifacts, generates an executive summary, and even points to in-house experts for follow-up. This retrieval-plus-synthesis loop has driven ~72% firm-wide adoption and saves teams ~30% of the time they once spent hunting through SharePoint, wikis, and email threads, proof that the decisive edge isn’t just a bigger model, but a meticulously engineered stream of proprietary context (Source).What Does ContextActuallyMean in the Enterprise?By now, it’s clear that providing the right context is key to unlocking the full potential of AI and agentic systems inside organizations. But “context” isn’t just a document or a code snippet; it’s a multi-layered, fragmented, and evolving ecosystem. In real-world settings, it spans everything from database schemas to team ownership metadata, each layer representing a different slice of what an intelligent system needs to reason, act, and adapt effectively.Based on my experience working across hundreds of data sources and collaborating with cross-functional product, engineering, and data teams, I’ve found that most enterprise context and information fall into nine broad categories. These aren’t just a checklist; they form a mental model: each category captures a dimension of the environment that AI agents must understand, depending on the use case, to operate safely, accurately, and effectively within your organization.Read the full article on Packt’s Medium. If you’re new, make sure to follow our Medium handle and subscribe to our newsletter for more insights like this!📈 Patterns & Practice: What’s Moving the World of Data & ML⭕ Implementing a Tool-Enabled Multi-Agent Workflow with Python, OpenAI API, and PrimisAI Nexus: Learn how to implement a multi-agent AI system using Python, OpenAI API, and PrimisAI Nexus. The tutorial covers setting up hierarchical supervision, defining structured JSON schemas, and integrating tools for code validation, statistical analysis, and documentation search. Agents collaborate to automate complex workflows across planning, development, QA, and data analysis with scalable, role-based coordination.⭕ Hugging Face Releases SmolLM3: A 3B Long-Context, Multilingual Reasoning Model: Hugging Face's SmolLM3 is a compact 3B-parameter multilingual model offering SoTA reasoning, tool use, and 128k-token context handling. Released in base and instruct variants, it rivals 7B+ models across benchmarks like XQuAD and MGSM. SmolLM3 is ideal for multilingual RAG, agent workflows, and edge deployments, delivering powerful performance with efficiency and accessibility.⭕ Microsoft Open-Sources GitHub Copilot Chat Extension for VS Code—Now Free for All Developers: Microsoft has open-sourced the GitHub Copilot Chat extension for VS Code under the MIT license, unlocking premium AI coding tools for free. With Agent Mode, Edit Mode, predictive Code Suggestions, and in-editor Chat, developers gain powerful automation, multi-file editing, and contextual assistance, paving the way for customizable, AI-enhanced workflows across open-source and enterprise environments.⭕ Google AI Just Open-Sourced a MCP Toolbox to Let AI Agents Query Databases Safely and Efficiently: Google’s new MCP Toolbox for Databases simplifies secure, schema-aware SQL integration for AI agents with just a few lines of Python. Part of the open-source GenAI Toolbox, it supports PostgreSQL/MySQL, MCP-compliant interfaces, connection pooling, and safe query generation, enabling reliable database access for LLM workflows in analytics, customer support, DevOps, and enterprise automation.⭕ The Five-Second Fingerprint: Inside Shazam’s Instant SongID: Part of the Behind the Tap series, this deep dive unpacks how Shazam identifies songs in seconds using audio fingerprinting, FFT-based spectrograms, and hash matching. It explains the journey from a tap to real-time song recognition, reveals Shazam’s scalable architecture, and explores its industry impact, from music discovery to market insights used by Apple and record labels.⭕ POSET Representations in Python Can Have a Huge Impact on Business: POSETs (Partially Ordered Sets) offer a powerful alternative to traditional ranking systems by preserving multidimensional relationships without forcing a linear order. This post shows how POSETs can improve decision-making by avoiding arbitrary weighting and oversimplification, using Python and the Wine Quality dataset to build dominance matrices, Hasse diagrams, and interpret incomparability across samples.⭕ Build Algorithm-Agnostic ML Pipelines in aBreeze: MLarena is a newly open-sourced, algorithm-agnostic machine learning toolkit built on MLflow for training, evaluating, tuning, and deploying models. It balances automation with expert control, offering built-in diagnostics, explainability tools, robust hyperparameter optimization via Bayesian search, and seamless MLflow integration. MLarena simplifies end-to-end ML workflows while enhancing model transparency, stability, and reproducibility.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0
Subscribe to Packt _DataPro
Get a weekly roundup of trending blogs, industry updates, and practical tutorials on Data Science, MLOps, and ML Algorithms in our Free Data & Machine Learning newsletter.

Merlyn from Packt
25 Jun 2025
Save for later

Microsoft Presidio, Amazon Bedrock + Arize Phoenix for Agent Observability, No-Code Forecasting with SageMaker Canvas

Merlyn from Packt
25 Jun 2025
Multi-Agent KYC with Google’s ADK, Inside MiniMax-M1: A New Long-Context RL FoundationBecome an AI Generalist that makes $100K (in 16 hours)AI isn’t the future — it’s the present, quietly reshaping work, money, and opportunity. McKinsey says AI is set to add $13Trillion to the economy by 2030 — but also replace millions of jobs. Will you use it to get ahead, or get left behind? Don’t worry here’s exactly what you need: Join the World’s First 16-Hour LIVE AI Mastermind for professionals, founders, consultants & business owners like you.Rated 4.9/5 by 150,000 global learners – this will truly make you an AI Generalist that can build, solve & work on anything with AI.In just 16 hours & 5 sessions, you will:✅ Learn the basics of LLMs and how they work.✅ Master prompt engineering for precise AI outputs.✅ Build custom GPT bots and AI agents that save you 20+ hours weekly.✅ Create high-quality images and videos for content, marketing, and branding.✅ Automate tasks and turn your AI skills into a profitable career or business.All by global experts from companies like Amazon, Microsoft, SamurAI and more. And it’s ALL. FOR. FREE. Join now and get $5100+ in additional bonuses: 🔥$5,000+ worth of AI tools across 3 days — Day 1: 3000+ Prompt Bible, Day 2: $10K/month AI roadmap, Day 3: Personalized automation toolkit.Attend all 3 days to unlock the cherry on top — lifetime access to our private AI Slack community!Register Now (free only for the next 72 hours)SponsoredSubscribe|Submit a tip|Advertise with usWelcome to DataPro 140 – Where Breakthrough AI Meets Practical Problem-SolvingTired of demos and theoretical fluff? From no-code forecasting to long-context AI, this week’s roundup dives into how today’s most compelling tools are reshaping what’s possible, without requiring you to reinvent your stack. Whether you're rethinking compliance with agentic workflows, streamlining data prep with natural language, or scaling models without breaking compute, these stories explore the friction points data teams face, and how smart engineering is solving them. Let’s get into what’s moving the space forward👇🔍 This Week’s Top Drops[Build AI Workflows with n8n + LLMs]Launch intelligent automations, daily briefs, customer bots, schedulers, without writing complex code.[Magenta RealTime: Music Meets LLMs]Google's open model lets you generate music live using SpectroStream and a transformer backbone.[MiniMax-M1: A 456B Long-Context Model]Crush reasoning bottlenecks with 1M-token context and lightning-fast attention, optimized for real-world use.[DSPy: Program AI, Don’t Just Prompt]Treat LLM workflows like code: structured logic, modules, and debug-ability built right in.[KYC Agents with Google’s ADK + Gemini]Skip the manual drudgery, automate onboarding with grounded search, sub-agents, and BigQuery.[Amazon Bedrock + Arize: Agent Observability]Gain full visibility into AI agent behavior, tool calls, and accuracy with production-grade insights.[Presidio for PII Detection + Hashing]Anonymize names, numbers, even custom IDs, safely, consistently, and at scale with Microsoft Presidio.[PyBEL for Bio Knowledge Graphs]Map disease pathways and protein interactions with this powerful toolkit for causal graph building.Whether you’re building agentic pipelines or anonymizing sensitive data, this week’s roundup proves you’re only ever a prototype away from production.Cheers,Merlyn ShelleyGrowth Lead, PacktJoin us on July 19 for a 150-minute interactive MCP Workshop. Go beyond theory and learn how to build and ship real-world MCP solutions. Limited spots available! Reserve your seat today.Use Code EARLY35 for 35% offTop Tools Driving New Research 🔧📊🔵 Building AI-Powered Low-Code Workflows with n8n: Discover how to automate personal and business tasks using n8n, a low-code platform with built-in AI. This blog walks through building three useful workflows: a daily briefing assistant, customer support bot, and appointment scheduler, while addressing prompt injection, memory setup, and alternatives for creating intelligent, efficient systems without heavy technical effort.🔵 google/magenta-realtime: Explore Magenta RealTime, Google’s open music generation model designed for real-time audio creation. Licensed under Apache 2.0 and CC-BY 4.0, it enables interactive music workflows using components like SpectroStream, MusicCoCa, and a transformer LLM. It supports live performance, education, and research, while outlining usage terms, risks, and limitations.🔵 tencent/Hunyuan3D-2.1: Get to know Hunyuan3D 2.1, a high-fidelity 3D asset generation framework from images, designed with production-ready PBR materials. Developed by Tencent, it builds on scalable diffusion models and supports text-to-3D and image-to-3D workflows. Backed by multiple arXiv publications, the project acknowledges open-source contributions and promotes reproducibility through public citation and benchmarking.🔵 MiniMaxAI/MiniMax-M1-80k: Tackle complex reasoning and long-context challenges with MiniMax-M1, a purpose-built open-weight model for data professionals. Designed with a 1M-token context window and lightning-efficient attention, it excels in software engineering, tool use, and advanced problem-solving, making it a reliable foundation for building next-gen AI applications in practical, high-stakes environments.Topics Catching Fire in Data Circles 🔥💬🔵 Data Has No Moat! Rethink data's role in the AI era. While powerful models grab headlines, this piece makes a compelling case for data as the true competitive moat. From poisoning risks to quality loops, it outlines why responsible, curated, and well-governed data is still the foundation of any trustworthy AI system that lasts.🔵 Agentic AI: Implementing Long-Term Memory. Build better LLM applications by implementing long-term memory, because short-term hacks won't scale. This piece breaks down practical strategies for data professionals, from hybrid search to knowledge graphs, and weighs open-source and vendor tools. It’s a clear guide for designing memory systems that reduce hallucinations and support reasoning over time.🔵 Programming, Not Prompting: A Hands-On Guide toDSPy. Move beyond fragile prompting with DSPy, a framework that treats LLM workflows like real programming. This hands-on guide shows how to build AI apps using DSPy modules, structure logic with signatures, and boost reliability through instruction optimization. For data professionals, it's a smarter way to design, debug, and scale GenAI systems.New Case Studies from the Tech Titans 🚀💡🔵 Amazon Bedrock Agents observability using Arize AI: Monitor and improve AI agents with the Amazon Bedrock–Arize Phoenix integration. Gain full traceability of agent decisions, evaluate tool call accuracy, and optimize performance with structured insights. This setup simplifies debugging, enhances reliability, and supports production-scale deployment, key for building transparent, efficient, and trustworthy generative AI applications end-to-end.🔵 No-code data preparation for time series forecasting using Amazon SageMaker Canvas: Prepare time series data without writing code using Amazon SageMaker Canvas and Data Wrangler. Import datasets, clean and transform data with natural language or visual tools, and resample for forecasting. With built-in security, validation, and modeling, this no-code workflow streamlines time series forecasting from raw CSV to predictive model in minutes.🔵 Gemini 2.5 Updates: Flash/Pro GA, SFT, Flash-Lite on Vertex AI: Build and scale confidently with Gemini 2.5 now on Vertex AI. Gemini 2.5 Flash and Pro are production-ready, with Flash-Lite and audio-capable Live API in preview. Get speed, reasoning, and fine-tuning for custom workflows. With full observability, multimodal depth, and real-world testimonials, this release levels up enterprise AI development.🔵 Build KYC agentic workflows with Google’s ADK: Streamline KYC with a multi-agent workflow using Google’s Agent Development Kit, Gemini models, Search Grounding, and BigQuery. This three-step guide shows how to orchestrate document checks, resume verification, and wealth analysis using agent tools and grounded search, boosting accuracy, automation, and auditability for financial institutions aiming to modernize compliance with AI.Blog Pulse: What’s Moving Minds 🧠✨🔵 Getting Started with Microsoft's Presidio: A Step-by-Step Guide to Detecting and Anonymizing Personally Identifiable Information PII in Text. Learn to detect and anonymize PII in free text using Microsoft Presidio. This hands-on guide walks through installing Presidio, recognizing standard and custom entities, applying anonymizers like hashing and reanonymization, and maintaining consistent outputs. With spaCy integration and reusable mappings, it’s a practical toolkit for responsible data handling in NLP workflows.🔵 A Coding Implementation for Creating, Annotating, and Visualizing Complex Biological Knowledge Graphs Using PyBEL. Use PyBEL to model complex biological systems like Alzheimer’s pathways through causal graph construction, network analysis, and custom visualization. This tutorial guides you through defining proteins and processes, analyzing node centrality, querying paths, and mining literature evidence, all in Google Colab, laying a strong foundation for biological knowledge graph exploration and enrichment.🔵 MiniMax AI Releases MiniMax-M1: A 456B Parameter Hybrid Model for Long-Context and Reinforcement Learning RL Tasks. MiniMax-M1 is a 456B open-weight hybrid model built for long-context and reinforcement learning tasks. With 1M-token context, lightning-fast attention, and efficient RL via the CISPO algorithm, it reduces compute cost while excelling in software engineering and agent tool use. A scalable, transparent breakthrough for real-world reasoning applications.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0

Merlyn from Packt
11 Jun 2025
Save for later

10,000x Faster Bayesian Inference, OpenAI on Countering Malicious AI, MCP integrations to Google Cloud Databases, MLOps Pipeline with Tekton and Buildpacks

Merlyn from Packt
11 Jun 2025
Gemini-Powered DataFrame Agent for Natural Language Data Analysis with Pandas and LangChain, VisualYour Exclusive Invite for the World’s first 2-day AI Challenge (usually $895, but $0 today)51% of companies have started using AITech giants have cut over 53,000 jobs in 2025 itselfAnd 40% of professionals fear that AI will take away their job.But here’s the real picture - companies aren't simply eliminating roles, they're hiring people who are AI-skilled, understand AI, can use AI & even build with AI. Join the online 2-Day LIVE AI Mastermind by Outskill - a hands-on bootcamp designed to make you an AI-powered professional in just 16 hours. Usually $895, but for the next 48 hours you can get in for completely FREE!In just 16 hours & 5 sessions, you will:Learn the basics of LLMs and how they workMaster prompt engineering for precise AI outputsBuild custom GPT bots and AI agents that save you 20+ hours weeklyCreate high-quality images and videos for content, marketing, and brandingAutomate tasks and turn your AI skills into a profitable career or businessKick off Call & Session 1- Friday (10am EST- 1pm EST)Sessions 2-5:Saturday 11 AM to 7 PM EST; Sunday 11AM EST to 7PM ESTAll by global experts from companies like Amazon, Microsoft, SamurAI and more. And it’s ALL. FOR. FREE. You will also unlock $3,000+ in AI bonuses: Slack community access, Your Personalised AI tool kit, and Extensive Prompt Library with 3000+ ready-to-use prompts - all free when you attend!Join in now, we have limited free seats!SponsoredSubscribe|Submit a tip|Advertise with usWelcome to DataPro #138 - Where AI Acceleration Meets Practical InsightThis week’s edition dives into the cutting edge of data science, AI tooling, and intelligent automation, highlighting breakthroughs that are reshaping how we build, reason, and scale.From a staggering 10,000x speed-up in Bayesian inference to OpenAI’s battle against malicious AI use, this issue captures the pulse of innovation across MLOps, LLM infrastructure, and trustworthy deployment. Google’s new MCP Toolbox integrations promise seamless AI-assisted development on Cloud Databases, while Tekton and Buildpacks simplify model automation with no Dockerfile in sight.We also explore research frontiers, from advanced molecular design powered by ether0’s RL-tuned 24B model, to VeBrain’s leap in embodied AI, letting language models perceive, reason, and act in physical environments. On the tooling side, Alchemist shows how to distill open datasets into generative gold, and Meta’s LlamaRL raises the bar on scalable RL fine-tuning for LLMs.Looking ahead, our preview spotlights a Gemini-powered Pandas agent capable of transforming natural language queries into statistical and visual insights, no code required. Plus, you’ll find a walkthrough on automating customer support with Bedrock and Mistral, and even a guide to running DeepSeek-R1 locally at home (if your GPU can handle it).SponsoredCloudVRM slashes vendor review and audit time by connecting directly to cloud environments, no spreadsheets, no forms, just real-time compliance, 24/7. Watch the demo.Whether you're in research, ops, or product, this editionoffers powerful perspectives and hands-on resources to keep your stack smart and future-ready.Cheers,Merlyn ShelleyGrowth Lead, PacktGet Chapter 1 of Learning Tableau 2025 – Free!Explore Tableau’s newest AI-powered capabilities with a free PDF of Chapter 1 from the latest edition of the bestselling series, Learning Tableau 2025.Written by Tableau Visionary Joshua Milligan, this hands-on guide helps you build smarter dashboards, master data prep, and apply AI-driven insights.Sign up to download your free chapter!Grab Your Free Chapter Now!Top Tools Driving New Research 🔧📊🔳ether0: A 24B LLM Trained with Reinforcement Learning RL for Advanced Chemical Reasoning Tasks. ether0 is a 24B-parameter language model developed by FutureHouse to tackle advanced chemical reasoning tasks. Trained using a blend of reinforcement learning and behavior distillation, it generates molecular structures as SMILES strings and significantly outperforms both general-purpose and chemistry-specific models. ether0 demonstrates exceptional accuracy and data efficiency, achieving 70% accuracy with only 60,000 training reactions, surpassing models trained on full datasets. Its architecture includes novel training strategies like GRPO, curriculum learning, and expert initialization, making it a new benchmark in scientific LLM development for molecular design and synthesis.🔳 OpenGVLab/VeBrain: Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces. Visual Embodied Brain (VeBrain) is a unified framework designed to extend multimodal large language models (MLLMs) into physical environments, enabling them to perceive, reason, and control in real-world spaces. By translating robotic tasks into text-based interactions within a 2D visual context, VeBrain simplifies multimodal objectives. It introduces a robotic adapter to convert MLLM-generated text into actionable control for physical systems. The accompanying VeBrain-600k dataset, meticulously curated with multimodal chain-of-thought reasoning, supports this integration. VeBrain significantly outperforms models like Qwen2.5-VL across multimodal and spatial benchmarks, and demonstrates superior adaptability and compositional reasoning in legged robot and robotic arm control tasks.🔳 Alchemist: Turning Public Text-to-Image Data into Generative Gold. Alchemist introduces a novel strategy for curating high-quality supervised fine-tuning (SFT) datasets to enhance text-to-image generation. By using a pre-trained generative model to identify impactful samples, the authors created a compact, diverse 3,350-sample dataset that significantly boosts the performance of five public T2I models. Unlike existing narrow-domain datasets, Alchemist is general-purpose and openly available, addressing limitations of proprietary data reliance. The approach offers a cost-effective and scalable alternative for dataset creation while improving image quality and stylistic variation in generative outputs. Fine-tuned model weights are also publicly released to support broader research and application.🔳 Meta Introduces LlamaRL: A Scalable PyTorch-Based Reinforcement Learning RL Framework for Efficient LLM Training at Scale. Meta’s LlamaRL is a new PyTorch-based framework designed to make reinforcement learning (RL) more scalable for training large language models. It uses an asynchronous, distributed architecture where components like generation and training run in parallel, reducing GPU idle time and improving memory efficiency. LlamaRL supports massive models, up to 405B parameters, with significant speedups, achieving over 10× faster RL step times compared to traditional methods. Features such as dedicated executors, NVLink-based synchronization, and offloading enable modularity and fine-grained parallelism. LlamaRL offers a flexible, high-performance infrastructure for aligning large models through RL at industrial scale.Topics Catching Fire in Data Circles 🔥💬🔳 Automate Models Training: An MLOps Pipeline with Tekton and Buildpacks. This tutorial introduces an automated MLOps pipeline for training GPT-2 models using Tekton and Buildpacks, without writing a Dockerfile. It demonstrates how to containerize training workflows and orchestrate CI/CD pipelines in Kubernetes. Using Buildpacks, the training code is converted into a secure container image, while Tekton Pipelines manages sequential tasks for building and executing training. A shared PersistentVolume ensures smooth data flow across steps. The pipeline is lightweight, reproducible, and perfect for integrating experimentation into production-grade ML workflows. This example highlights the growing importance of efficient, code-light automation in model development.🔳 Prescriptive Modeling Unpacked: A Complete Guide to Intervention with Bayesian Modeling. This guide explores how prescriptive modeling, using Bayesian methods, enables data-driven intervention in complex systems rather than just prediction. Moving beyond forecasting, it identifies causal drivers in systems and quantifies the effects of changes. With hands-on examples in predictive maintenance and Bayesian networks via the bnlearn Python library, the article walks through building causal models, inferring interventions, and applying them to real-world scenarios like water infrastructure. It also covers structure learning, synthetic data generation, and practical cost-benefit considerations, making it a comprehensive resource for actionable analytics in operations and engineering.🔳 How OpenAI responding to The New York Times’ data demands in order to protect user privacy? OpenAI is actively resisting a legal demand from The New York Times to indefinitely retain ChatGPT and API user data, a move it argues undermines its privacy commitments. The order excludes Enterprise and Zero Data Retention API users. OpenAI is appealing the decision, maintaining data will remain securely stored, restricted to legal teams, and used only to meet legal obligations. Deleted chats, normally erased within 30 days, are affected by the hold, but OpenAI vows to fight further access requests and uphold user privacy throughout the legal process. Training policies and business data protections remain unchanged.🔳 What execs want to know about multi-agentic systems with AI? This field report highlights key lessons from enterprise adoption of Multi-Agent Systems (MAS). While MAS can transform complex processes through coordinated AI agents, many leaders limit its value by simply automating legacy workflows. Success requires reimagining processes, designing thoughtful agent collaboration, and embedding governance and ethics from the start. Common missteps include neglecting collaboration logic, delaying ethical safeguards, and underestimating the shift needed to harness MAS fully. Executives most often ask how to measure ROI beyond cost, how to balance human and AI roles, and how to manage ethical risks. Effective MAS design relies on clear goals, rigorous testing, and human-AI orchestration.New Case Studies from the Tech Titans 🚀💡🔳 10,000x Faster Bayesian Inference: Multi-GPU SVI vs. Traditional MCMC. Bayesian inference has traditionally been limited by high computational demands, especially in large-scale applications. This guide demonstrates how Stochastic Variational Inference (SVI) on multi-GPU setups can dramatically accelerate Bayesian modeling, achieving up to a 10,000x speedup over traditional CPU-based MCMC. Using JAX and NumPyro, data is efficiently sharded and replicated across GPUs, enabling scalable inference for millions of observations and parameters. Benchmarks show multi-GPU SVI reduces training time from days to minutes, making large hierarchical Bayesian models feasible for production. This approach is ideal for practitioners seeking rapid, scalable, and approximate Bayesian solutions in real-world settings.🔳 BenchmarkQED: Automated benchmarking of RAG systems:BenchmarkQED is an automated benchmarking suite designed to rigorously evaluate retrieval-augmented generation (RAG) systems. Developed to support tools like GraphRAG, it includes components for query generation (AutoQ), evaluation (AutoE), and dataset structuring (AutoD). BenchmarkQED enables consistent testing across local-to-global query types, using synthetic queries and LLM-based judgments. LazyGraphRAG, evaluated with this suite, consistently outperforms traditional and advanced RAG methods, even those with massive 1M-token contexts, across comprehensiveness, diversity, empowerment, and relevance. BenchmarkQED and its datasets, now open-source, offer a scalable, structured path for testing next-gen RAG capabilities in real-world QA applications.🔳 OpenAI on Countering Malicious AI – June 2025 OpenAI’s June 2025 report highlights how its teams are actively detecting and disrupting malicious uses of AI. In line with its mission to ensure AI benefits humanity, the company outlines efforts to block harmful applications such as cyber espionage, social engineering, scams, and influence operations. By leveraging AI to augment internal investigative teams, OpenAI has rapidly identified and neutralized threats over the past three months. The report reinforces the importance of democratic AI governance and common-sense safeguards to prevent misuse by authoritarian regimes and bad actors while supporting global safety and accountability.🔳 Deploying Llama4 and DeepSeek on AI Hypercomputer: Google has released new optimized recipes for deploying Meta’s Llama4 and DeepSeek models using its AI Hypercomputer platform. These guides streamline the setup of powerful MoE-based LLMs like Llama-4-Scout and DeepSeek-R1 across Trillium TPUs and A3 GPUs. Using inference engines like JetStream, MaxText, vLLM, and SGLang, developers can now efficiently run large models with multi-host support, minimal configuration, and reproducible performance. Recipes cover tasks such as model checkpoint conversion, TPU/GPU provisioning, and benchmarking (e.g., MMLU), enabling scalable, high-throughput inference for cutting-edge open-source LLMs in production-grade environments.🔳 New MCP integrations to Google Cloud Databases: Google Cloud has announced new MCP Toolbox integrations for databases, designed to supercharge AI-assisted development. The open-source Model Context Protocol (MCP) server now supports seamless connections between AI coding assistants (like Claude Code, Cline, and Cursor) and databases such as BigQuery, AlloyDB, Cloud SQL, Spanner, and others. These new capabilities enable developers to perform tasks like schema design, data exploration, code refactoring, and integration testing using natural language prompts within their IDEs. The result: faster, smarter development workflows, with AI handling the SQL and schema logic, dramatically reducing setup and iteration time.Blog Pulse: What’s Moving Minds 🧠✨🔳 Mastering SQL Window Functions: Mastering SQL Window Functions offers a clear and practical introduction to using window functions for powerful row-wise analysis without collapsing data. Unlike traditional aggregations, window functions (like SUM() OVER or RANK() OVER) preserve individual records while enabling calculations across partitions. Examples include calculating totals per brand, ranking by price, and computing year-wise averages, all while retaining full row-level detail. These functions are essential for tasks like ranking, comparisons, and cumulative metrics, making them a vital tool in modern analytics workflows. However, they may incur performance costs on large datasets, so use them judiciously.🔳 Automate customer support with Amazon Bedrock, LangGraph, and Mistral models: This walkthrough demonstrates how to build an intelligent, multimodal customer support workflow using Amazon Bedrock, LangGraph, and Mistral models. By combining large language models with structured orchestration and image-processing capabilities, the solution automates tasks such as ticket categorization, transaction and order extraction, damage assessment, and personalized response generation. LangGraph enables complex, stateful agent workflows while Amazon Bedrock provides secure, scalable access to LLMs and Guardrails for responsible AI. With integrations for Jira, SQLite, and vision models like Pixtral, this framework delivers real-time, context-aware support automation with observability and safety built in.🔳 Run the Full DeepSeek-R1-0528 Model Locally: DeepSeek-R1-0528, a powerful reasoning model requiring 715GB of disk space, is now runnable locally thanks to Unsloth's 1.78-bit quantization, reducing its size to 162GB. This guide explains how to deploy the quantized version using Ollama and Open WebUI. With at least 64GB RAM (CPU) or a 24GB GPU (for better speed), users can serve the model via ollama run, launch Open WebUI in Docker, and interact with the model through a local browser. While GPU usage offers ~5 tokens/sec, CPU-only fallback is much slower (~1 token/sec). Setup is demanding, but viable with persistence.🔳 How to Build an Asynchronous AI Agent Network Using Gemini for Research, Analysis, and Validation Tasks? The Gemini Agent Network Protocol offers a modular framework for building cooperative AI agents, Analyzer, Researcher, Synthesizer, and Validator, using Google’s Gemini models. This tutorial walks through creating asynchronous workflows where each agent performs role-specific tasks such as breaking down complex queries, gathering data, synthesizing information, and verifying results. By using Python's asyncio for concurrency and google.generativeai for model interaction, the network dynamically routes tasks and messages. With detailed role prompts and shared memory for dialogue context, it allows for efficient multi-agent collaboration. Users can simulate scenarios such as analyzing quantum computing’s impact on cybersecurity and observe real-time agent participation metrics.🔳 Build a Gemini-Powered DataFrame Agent for Natural Language Data Analysis with Pandas and LangChain: This tutorial demonstrates how to combine Google’s Gemini models with Pandas and LangChain to create an intelligent, natural-language-driven data analysis agent. Using the Titanic dataset as a case study, the setup allows users to query the data conversationally, eliminating the need for repetitive boilerplate code. The Gemini-Pandas agent can answer simple questions such as dataset size, compute survival rates, or identify correlations. It can also handle advanced analyses like age-fare correlation, survival segmentation, and multi-DataFrame comparisons. Custom analyses, such as building passenger risk scores or evaluating deck-wise survival trends, are also supported. With just a few lines of Python and LangChain tooling, analysts can turn datasets into a conversational playground for insight discovery.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 1
  • 0

Merlyn from Packt
11 Sep 2025
Save for later

GibsonAI Memori: SQL-Native Memory for Agents, NVIDIA’s Universal Deep Research, Conversational Commerce Agent on Vertex AI

Merlyn from Packt
11 Sep 2025
Free eBook: Debugging Apache Airflow® DAGsFree eBook: Fix your Airflow DAG errors fasterEven the most advanced Airflow users encounter DAG errors and task failures. That’s why we wrote Debugging Apache Airflow® DAGs. It’s a guide written by practitioners, for practitioners covering everything you need to know to solve issues with your DAGs:✅ Identifying issues during development✅ Using tools that make debugging more efficient✅ Conducting root cause analysis for complex pipelines in productionGET YOUR FREE GUIDE NOWSponsoredSubscribe|Submit a tip|Advertise with UsWelcome to DataPro 149- yourgo-to newsletter for all things Data and AI.This edition is packed with breakthroughs, experiments, and tutorials that show how fast the AI + data stack is evolving. From SQL-native memory engines to federated AI registries, adaptive defenses in federated learning, and even a 1950s algorithm powering computer vision, the highlights are designed to spark both curiosity and practical takeaways.Here’swhatyou’lldiscover 👇🔹MCP Registry Preview: DNS for AI Context-Meet the federated system for discovering AI servers, designed to scale like the internet itself.🔹Is Your Training Data Representative? PSI & Cramér’s V in Python- Learn how to measure representativeness, automate comparisons, and catch dataset drift before it breaks your models.🔹Fighting Back Against Attacks in Federated Learning-See how poisoning attacks work, why existing defenses fall short, and how adaptive strategies like EE-Trimmed Mean change the game.🔹Top 7 MCP Servers for Vibe Coding- From Git integration to browser automation and memory layers, these servers unlock context-rich collaboration between developers and AI agents.🔹NVIDIA’s Universal Deep Research (UDR)-A prototype framework that separates research strategy from the LLM itself, making deep research scalable, auditable, and customizable.🔹GibsonAIMemori: SQL-Native Memory for Agents-Forget costly vector DBs: this open-source memory engine makes agent memory transparent, portable, and cheap to run.Each story blendscutting-edgeideas with hands-on value,perfect for anyone building smarter AI systems, securing their pipelines, or just keeping ahead of the curve.So, without further ado, let’s jump in.Cheers,Merlyn ShelleyGrowth Lead, PacktTop Tools Driving New Research 🔧📊🔸MCP Team Launches the Preview Version of the 'MCP Registry': A Federated Discovery Layer for Enterprise AI.This blog unpacks the MCP Registry, a new open-source system designed as “DNS for AI context.” It explains why the federated model beats a single registry, how it secures enterprise AI, and what makes it scalable.You’llalso find details on its architecture, governance, and open-source foundation, plus practical FAQs for getting started with the preview release.🔸Building Advanced MCP (Model Context Protocol) Agents with Multi-Agent Coordination, Context Awareness, and Gemini Integration.Advanced MCP Agents can now be built and run insideJupyterorColabwith practical features like multi-agent coordination, context awareness, and Gemini integration. This tutorial shows how role-based agents such as researchers, analyzers, and executors work together as a swarm,maintainmemory for continuity, and deliver coherent results for complex, real-world AI tasks.🔸Is Your Training Data Representative? A Guide to Checking with PSI in Python:Checkingif your training data trulyrepresentsreality matters at build, deploy, andmonitorstages. This guide shows how to compare samples with PSI and Cramér’s V, from visual checks to robust stats, then automates the workflow in Python and exports an Excel report.You’llsee a worked example on Communities & Crime and clear thresholds for action.🔸Fighting Back Against Attacks in Federated Learning:Federated learning promises privacy-preserving training, but it also opens the door to subtle attacks like data poisoning and model manipulation. In this project, a multi-node simulator built onFEDnexplores how such attacks work, how currentdefenceshold up, and why adaptive strategies like EE-Trimmed Mean are needed. Experiments reveal lessons for making FL more resilient and trustworthy.Topics Catching Fire in Data Circles 🔥💬🔸Top 7 Model Context Protocol (MCP) Servers for Vibe Coding:Model Context Protocol servers areemergingas the backbone of Vibe Coding, where developers and AI agents collaborate in real time. This guide highlights seven standout MCP servers,from Git integration and live database access to browser automation, persistent memory, multi-agent orchestration, and research support,that make coding more adaptive, reproducible, and context-rich for modern development workflows.🔸How to Build a Complete End-to-End NLP Pipeline with Gensim: Topic Modeling, Word Embeddings, Semantic Search, and Advanced Text Analysis.An end-to-end NLP pipeline can be built inGensimthat covers preprocessing, topic modeling, embeddings, similarity search, and advanced analysis. This tutorial shows how to run it all inColab, from Word2Vec training and LDA topic modeling to coherence evaluation, visualization, and document classification. The result is a reusable framework for exploring and interpreting text data at scale.🔸Understanding the BigQuery column metadata (CMETA) index:BigQueryis pushing beyond petabyte-scale warehouses to petabyte-scale tables, where even metadata becomes big data. To keep queries fast and efficient, Google introduced the Column Metadata (CMETA) index, an automated, zero-maintenance system that prunes blocks early, saving time and slots. This blog explains how CMETA works, its impact on performance, and how to maximize its benefits.🔸When A Difference Actually Makes A Difference:A five-point gap on a bar chart can meanvery differentthings depending on variance, sample size, and effect size. In this bite-sized guide, Mena Wang shows business leaders how to look beyond averages, use statistical tests, and weigh effect sizes before acting. The lesson: not every “significant” difference is worth millions in investment.New Case Studies from the Tech Titans 🚀💡🔸NVIDIA AI Releases Universal Deep Research (UDR): A Prototype Framework for Scalable and Auditable Deep Research Agents.NVIDIA’s Universal Deep Research (UDR) is a prototype framework that separates research strategy from the underlying LLM, making deep research flexible, auditable, and scalable. Unlike rigid model-bound tools, UDR lets users design custom workflows, enforce validation rules, and swap models. With templates like Minimal, Expansive, and Intensive, UDR enables transparent, cost-efficient research pipelines for science, enterprise, and startups.🔸GKE Inference Gateway and Quickstart are GA:Google Cloud is expanding its AIHypercomputerstack with new inference capabilities in GKE Inference Gateway, now generally available. Highlights include prefix-aware routing for up to 96% faster TTFT, disaggregated serving for 60% higher throughput, and Anywhere Cache for 4.9x faster model loads. Paired with GKE InferenceQuickstart, teams can benchmark,optimize, and deploy LLM inference stacks in days instead of months.🔸Announcing Dataproc multi-tenant clusters:Google Cloud is introducingDataprocmulti-tenant clusters, giving data science teams a shared notebook environment that balances efficiency with strong isolation. Instead of siloed resources or weak security, admins can map users to service accounts, enforce IAM policies, and scalecomputedynamically. WithJupyterintegration via Vertex AI Workbench or third-party setups, teams get faster collaboration, lower costs, and enterprise-grade control.🔸Exploring Merit Order and Marginal Abatement Cost Curve in Python:This tutorial shows how to use Python to model electricity pricing anddecarbonisation. First, it builds a merit order curve to show how different power plants, ordered by cost, set the market price. Then it introduces a Marginal Abatement Cost Curve to comparedecarbonisationoptions by cost and impact. The code includes interactive charts to explore scenarios easily.Blog Pulse: What’s Moving Minds 🧠✨🔸GibsonAI Releases Memori: An Open-Source SQL-Native Memory Engine for AI Agents.GibsonAIhas releasedMemori, an open-source SQL-native memory engine for AI agents. Instead of relying on costly, opaque vector databases, Memori uses standard SQL (SQLite, PostgreSQL, MySQL) to provide persistent, transparent, and auditable memory. With a single line of code, agents gain context retention across sessions, reducing redundancy, cutting infrastructure costs by up to 90%, and giving users full control over their data.🔸Introducing Conversational Commerce agent on Vertex AI:Google Cloud has launched theConversational Commerce agent, now generally available in Vertex AI, to help retailers meet the shift toward longer, more complex search queries. Powered by Gemini, it enables natural, back-and-forth shopping conversations that guide users from discovery to checkout. Early adopters like Albertsons are seeing customers add more items to their carts, boosting sales through smarter, more intuitive product discovery.🔸Automate app deployment and security analysis with new Gemini CLI extensions:Google just introduced two newGemini CLIextensions that bring security and deployment right into your terminal. With/security:analyze, you can scan code for vulnerabilities locally (and soon in GitHub PRs) with clear, actionable fixes.With/deploy, you can ship apps directly toCloud Runin one simple command.It’sthe start of a broader, extensible Gemini CLI ecosystem.🔸The Hungarian Algorithm and Its Applications in ComputerVision:TheHungarian algorithm, first developed in the 1950s, is a powerful way to solve assignment problems, optimally matching tasks to workers, or objects across video frames. In computer vision, it underpinsmulti-object trackingby minimizing distances between bounding boxes detected in consecutive frames. This ensures consistent object tracking, even in complex scenes with motion, occlusion, or overlapping detections.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0

Merlyn from Packt
22 Jan 2026
Save for later

The State of Apache Airflow 2026

Merlyn from Packt
22 Jan 2026
What hands-on AI-102 preparation really looks likeThe State of Apache Airflow2026We asked 5,800 data engineers about how they're using Airflow in 2026. Find out what they had to say.Join Airflow committer Vikram Koka and other experts on February 5 to learn:Latest trends in how teams are using Airflow todayEmerging use cases from the annual Airflow surveyHow to make the most of Airflow for data orchestration in 2026Join us to hear directly from leaders in the community and discover how to get the most out of Airflow in the year ahead. February 5, 2026 | 11am ET / 4pm GMT.Reserve My SpotSponsoredWelcome to DataPro #163. This week, we’re introducing a new format with a hands-on case study from Microsoft Senior Solution Architect Peter T. Lee. Using a real multi-agent Azure AI system for student loan processing, the piece shows how practical projects can support Azure AI-102 preparation while building real-world system design skills.This case study focuses on what matters most when building AI systems in practice: orchestration, document intelligence, and clear decision-making. You’ll see how multi-agent design helps turn messy PDFs into explainable workflows, and why this approach is valuable not just for certification prep, but for anyone working with data and applied AI. Dive in to see how it all comes together.Cheers,Merlyn ShelleyGrowth Lead, PacktSubscribe|Submit a tip|Advertise with UsBook Now & Save 30% [Use code FINAL30]From Exam Prep to Production: Building a Multi-Agent Student Loan Assistant on AzureThe real problem: intelligence without orchestrationModern AI systems rarely fail because the models are weak. They fail because the surrounding system is poorly designed. Data is fragmented across documents, decisions are opaque, workflows are brittle, and human review is bolted on as an afterthought. This becomes especially visible in document-heavy financial workflows such asstudent loan processing.A typicalstudent loan application still depends on static web forms, manual document checks, and backend rules that are difficult to explain or adapt. Applicants upload PDFs, operations teams reconcile mismatched data, and decisions take days or weeks to arrive. From a data and AI perspective, the core issue is not intelligence but orchestration: it's how unstructured data, business logic, and decisions are connected in a controlled, auditable way.Reframing the workflow as a conversational systemTheMulti-Agent Student Loan Assistantaddresses this problem by rethinking loan processing as a conversational, agent-driven workflow. Instead of forcing users through rigid forms, the system allows applicants to interact naturally, upload documents directly in chat, review extracted data, and receive a clear, explainable decision.The emphasis is not automation for its own sake. It is about building structured intelligence where users remain in the loop, data is validated before decisions are made, and outcomes can be explained in plain language.Book Now & Save 50% [Use code EARLY50]Why multi-agent architecture mattersRather than relying on a single, monolithic AI model, the system is built around a multi-agent architecture. Different agents are responsible for conversation handling, document extraction, cross-document validation, and decision-making, coordinated through an orchestration layer.Crucially, business-critical logic such as debt-to-income calculations and interest rate evaluation is handled outside the language model. This separation between probabilistic AI and deterministic computation is what makes the system testable, governable, and suitable for regulated environments.For data professionals, this is the central architectural lesson: real-world AI systems demand decomposition, clear responsibilities, and deliberate boundaries between reasoning and rules.Document intelligence as a practical data problemPDFs remain one of the most common and problematic data formats in enterprise workflows. The student loan assistant demonstrates how OCR, structured extraction, and cross-document validation can be combined into a single, coherent process.Equally important is the human confirmation step. Extracted data is reviewed before automation proceeds, reinforcing trust and reducing downstream errors. This pattern is increasingly important as organizations move generative AI into operational systems.Why this matters for Azure AI-102 certification prepTheAzure AI-102 examincreasingly tests architectural thinking rather than isolated service knowledge. Candidates are expected to understand how Azure OpenAI, deployment choices, identity, containers, and operational constraints fit together in real solutions.This project mirrors that reality. It forces you to reason about service integration, workflow design, responsible AI considerations, and deployment trade-offs. The same mental models that help you understand this system also help you navigate scenario-based exam questions.How the book and this project help you learnThis case study follows the same learning approach used throughoutAzure AI-102 Certification Essentials. The book is written by Peter T. Lee, a Senior Solution Architect at Microsoft, with over 25 years of experience working with data platforms, cloud systems, and applied AI. Rather than focusing only on theory or exam definitions, the book is built around real scenarios drawn from practice. Projects like the Multi-Agent Student Loan Assistant show how Azure AI services are combined to solve realistic problems, helping readers understand not just what a service does, but when and why to use it.For learners, this creates a clear connection between certification study and hands-on work. The same concepts you revise for the exam are reinforced by building and inspecting real systems.What this means for data and BI professionalsFor data analysts, BI practitioners, and engineers moving toward applied AI roles, the shift is already underway. AI is no longer limited to dashboards or isolated models. It increasingly shows up as intelligent workflows that work with documents, validate data across sources, and support decisions in a transparent way.TheMulti-Agent Student Loan Assistantis best viewed as a learning example rather than a finished product. By exploring how it is designed and deployed, readers gain experience that carries over to bothAzure AI-102 exam scenarios and real project work. This blend of understanding and practice is what helps Azure AI skills hold up beyond the certification itself.A practical next stepFor many people preparing for Azure AI-102, the challenge is not finding information, but knowing how to connect it into a working system. Case studies like the Multi-Agent Student Loan Assistant help bridge that gap by showing how individual Azure AI services behave once they are combined, deployed, and constrained by real-world requirements.If you are studying for the certification, exploring projects like this alongsideAzure AI-102 Certification Essentialscan help ground abstract topics in concrete examples. The book provides structured guidance through the exam objectives, while hands-on systems like this one help you test your understanding against realistic design decisions.Taken together, they offer a way to prepare that is less about memorization and more about developing the judgment needed to design, build, and reason about AI systems on Azure, both in an exam setting and beyond it.If you want to explore this learning approach in more depth, you cancheck outAzure AI-102 Certification Essentialshere.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0

Merlyn from Packt
05 Feb 2026
Save for later

From The Kaggle Book to Designing ML and GenAI Systems That Scale

Merlyn from Packt
05 Feb 2026
Luca Massaron on Kaggle mastery, plus Sairam Sundaresan on scalable ML and GenAI systems.Packt and Go1 Invite You to Shape a New Study on Developer LearningAs AI generates more learning content, it is becoming harder to see where expert input really makes a difference. Packt has recently partnered with Go1 to create a short study looking at how developersactually learntoday, and when structured courses still matter alongside AI tools.If you work with learning or rely on it to build skills, your perspective would be useful. The survey takes under5minutestocomplete,and the results will be sharedin a study published in March.Take the 5-Minute SurveySubscribe|Submit a tip|Advertise with UsWelcome to DataPro #164. As data science and ML roles continue to evolve, standing out increasingly depends on how well you can demonstrate real-world problem solving, not just model knowledge.So, if you’ve ever wondered whether Kaggle is “just competitions,” or why so many strong data scientists and ML engineers still credit it for major career breakthroughs, this issue is for you.In this edition, Luca Massaron, co-author of The Kaggle Book, 2nd Edition, breaks down what Kaggle truly offers beyond leaderboards. He explains how notebooks, datasets, and competition workflows help build a visible record of problem solving, experimentation, and technical judgment. More importantly, he shows how this experience translates into real-world skills and interview-ready stories using the STAR framework.To support your learning, our authors have also created a free reference cheatsheet that maps all the libraries covered in the book, giving you a clear learning path as you work through the resources. You can download it here:Kaggle Book Cheatsheet.💡 Workshop Spotlight: Machine Learning and Generative AI System Design WorkshopJoin Sairam Sundaresan, AI Engineering Leader, for a hands-on system design workshop on February 28, focused on building machine learning and generative AI systems that actually scale. In this live session, you’ll move beyond model demos and learn how experienced architects design end-to-end AI systems by balancing cost, latency, quality, and risk.Through guided exercises and design sprints, you’ll practice making real architectural trade-offs and defining success metrics that go beyond accuracy. Whether you’re an ML engineer, architect, or product leader, this workshop equips you with reusable frameworks to design AI systems that hold up in production and evolve with changing models and regulations.Register Now and Save 35%Use DATAPRO35 at checkout for early access savings and reserve your seat.Cheers,Merlyn ShelleyGrowth Lead, PacktThe Essential Asset: Leveraging Kaggle Experience in a Competitive Professional LandscapeEngaging with the Kaggle platform offers a clear advantage for data science professionals seeking to emerge in complex and challenging job market situations, such as the recent one marked by widespread layoffs and hiring difficulties. While competitive data science does not cover the entire span of enterprise-level processes related to data processing and MLOPs, the knowledge and skills acquired on Kaggle are a significant complement to real-world experience. Kaggle serves as an integrated environment for acquiring, documenting, validating, and showcasing competencies that help candidates stand out from the crowd and avoid becoming obsolete in front of automated machine learning (AutoML) or other off-the-shelf solutions, such as recent tabular AI solutions.The Creation of a Verifiable PortfolioEmployers often view a robust portfolio of projects as a great demonstration of technical knowledge and hands-on experience compared to academic credentials alone. Kaggle facilitates the building up of this critical professional asset.First of all, Notebooks are recognized as the most important tool (after rankings) for demonstrating a candidate's abilities, providing tangible evidence of their capacity for clean coding and effective communication. Even if top ranks are not achieved, high-quality Notebooks focused on Exploratory Data Analysis (EDA), tutorials on model architectures, or implementations of cutting-edge research prove crucial abilities, such as extracting visual and non-visual insights from data. Notebooks showcase not justwhat a candidate has done, but also how they approach problems and communicate insights and conclusions, which is critical for working with management, clients, and experts from diverse backgrounds in a business-oriented company environment.On the other hand, Kaggle Datasets provide an excellent means for demonstrating ability with data to be used with Machine Learning (ML) algorithms. By curating, cleaning, and documenting data, professionals can publish and maintain a dataset on Kaggle, thereby demonstrating their understanding of the data's value and potential. The presence of a description, tags, a license, sources, and a frequency of updates are pieces of information used to calculate a usability index, which helps others understand how to use the data. This shows an ability to manage and document data over time. Recently, this opportunity has also been extended to models, allowing the showcase of the necessary competencies in maintenance, fine-tuning, and evaluation of both small and large language models.This Week’s Sponsor: Progress TelerikWebinar: How to Build Faster with AI AgentsLearn how full-stack developers boost productivity by up to 50% using AI agents to automate layout, styling, and component generation with RAG and LLM pipelines.See how orchestration and spec-driven workflows keep quality and consistency in check. Save your seat.Accelerated Skill Acquisition and MarketabilityKaggle participation fosters self-development, exposing data scientists to diverse data types and problems, demanding rapid iteration on model hypotheses, and requiring extensive feature engineering, experience akin to "competition heat." This challenging environment sharpens skills necessary for finding quick and effective solutions to data problems.For job seekers, this translates directly into marketability. Recruiters and human resource departments often monitor Kaggle profiles and rankings when searching for candidates with specific or rare competencies, such as those demonstrated in NLP or computer vision competitions. Consistently good performance in multiple competitions signals a genuine competency and provides verifiable credentials that differentiate an applicant from the crowd.Furthermore, teaming up in competitions teaches individuals to work collaboratively toward a common goal within a limited time frame, and teamwork is a highly valued quality in data science teams. Participating in Kaggle competitions also enhances networking opportunities, facilitating connections that may result in job referrals and opportunities.The Kaggle Book, 2nd Edition isn’t just about winning competitions. For data scientists and ML engineers, it offers a practical way to deepen modeling intuition, experiment with real world datasets, and refine end to end problem solving through notebooks and iterative workflows.As data science roles increasingly demand production awareness, rapid experimentation, and clear communication of results, this book helps you build skills that translate directly into stronger models and better technical decisions. With 30% off the eBook and 20% off the print edition, it’s a timely opportunity to add a structured, hands on reference to your learning stack.Add to CartTranslating Experience into Interview Gold via the STAR ApproachThe experience gained on Kaggle is invaluable during the job interview process. Candidates should leverage their competition efforts to demonstrate problem-solving capabilities using the STAR (Situation, Task, Action, Result) approach. This approach requires structuring competition narratives to emphasize past behavior rather than simply reciting technical capabilities.For example, when detailing a challenging competition:Situation: The candidate must provide a clear context for the problem encountered, detailing the environment and why the situation required attention or action.Task: Clearly explain the objective taken on, such as cleaning messy data, doing explorative analysis (EDA), or continuously improving a benchmark model.Action: Describe the specific steps executed. This can involve explaining the methodologies or media (such as notebooks) used to implement the solution.Result: Articulate the achievement, whether it was improving business value, beating a reference benchmark, or learning from the challenges faced.By utilizing the STAR framework with Kaggle examples, professionals can craft compelling narratives that effectively articulate their problem-solving capabilities and capacity for incremental improvements, thus granting them an edge over other applicants in a very competitive hiring landscape.Wrapping up how Kaggle can give a boost to your careerThe ability to build a robust portfolio, rapidly acquire new skills, and articulate experiences with clarity and confidence are timeless assets in any competitive field. Kaggle provides a unique and effective arena for developing these very competencies. The platform's emphasis on tangible results and peer-reviewed work ensures that the skills showcased are not merely theoretical but demonstrably real. For professionals committed to lifelong learning and staying ahead of the curve, engaging with Kaggle is a direct investment in their career longevity and relevance. By translating this experience into compelling narratives, as outlined through the STAR approach, candidates can effectively communicate their value, demonstrating that they are not just spectators in the data science field but proactive actors in its evolving future.Reading along? Don’t forget to download the free Kaggle Book Cheatsheet a clear reference point for the libraries covered throughout the book.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0

Merlyn from Packt
19 Feb 2026
Save for later

Vertex AI Provisioned Throughput, Bedrock AgentCore, Claude 4.6, GLM-5 & Risk-Based Optimisation for Production-Grade GenAI Systems

Merlyn from Packt
19 Feb 2026
Designing AI systems that survive scale.👋 Hello ,Welcome to DataPro #165Stop Forecasting. Start Designing for Uncertainty.In a recent Packt live workshop led by Professor Gerhard Kling of the University of Aberdeen, a powerful idea surfaced. The most fragile component in most optimisation systems is the variable we try hardest to predict. In finance, that is expected returns. In AI, it is model performance. When regimes shift, correlations spike, or data drifts, systems built on unstable assumptions break. Risk-based portfolio optimisation offers a different approach. Optimise around structure rather than speculation. Focus on covariance, diversification, and robustness. For data scientists, the lesson extends far beyond markets. In unstable systems, architecture matters more than forecasts. The session, part of Packt’s ongoing live workshop series and hosted by Abhishek Kaushik, reinforced this principle clearly.This week’s developments in ML and GenAI reflects the same shift. From guaranteed provisioned throughput on Vertex AI to Amazon’s production-grade agent evaluation frameworks, the industry conversation is moving from model benchmarks to system durability. That is why Packt’s upcoming Machine Learning and Generative AI System Design Workshop on Feb 28 from 9:30 AM to 2:00 PM EDT stands out. Led by AI Engineering Leader Sairam Sundaresan, this four-hour hands-on session focuses on end-to-end system design. You will work through architecture trade-offs across cost, latency, evaluation metrics, governance, and long-term adaptability. If you are responsible for systems that must scale, evolve, and survive model shifts, this workshop is built for you. Use code DATAPRO35 for 35 percent off. If production-grade AI architecture is part of your roadmap this year, this is a practical place to begin.Here is what stood out this week in data science and machine learning.Risk-Based Optimisation in Python: Lessons from Professor Gerhard Kling’s Live WorkshopMachine Learning & Generative AI System Design Workshop (Use codeDATAPRO35for 35% off)Vertex AI Provisioned Throughput: Guaranteed capacity for enterprise agentsAmazon BedrockAgentCoreand unified enterprise intelligenceEvaluating AI Agents in Production: Amazon’s real-world frameworkFrontier Model Updates: Claude 4.6, GLM-5, MiniMax-M2.5, Qwen3.5,PersonaPlexRead through to the end for a deeper breakdown of what these shifts mean for production ML systems and how to design architectures that actually hold under pressure.📢 Important Update: DataPro Is Moving to SubstackStarting next week, DataPro will transition fully to Substack. From 25 February 2026, all issues will be delivered from packtdatapro1@substack.com.To ensure uninterrupted delivery, please whitelist this email address in your mail client. No other action is required.You will continue receiving the newsletter on the same weekly schedule. On Substack, you will also gain access to more expert-led content and greater control over your subscription preferences. You’ll be able to like, comment, and share your views directly within the newsletter, all in one place.We hope this new platform becomes a stronger space for thoughtful discussions, shared insights, and your ideas.Cheers,Merlyn ShelleyGrowth Lead, PacktSubscribe|Submit a tip|Advertise with UsYou Can’t Predict Returns. So Why Are You Optimising Them?If you’ve ever built a model that looked perfect in-sample and disappointing in production, you already understand the core problem in portfolio optimisation.Finance has the same issue.For decades, we’ve taught mean-variance optimisation as the foundation of portfolio construction. Estimate expected returns. Estimate covariances. Optimise. Trace the efficient frontier.It is mathematically elegant.It is also fragile.In a recent live workshop led by Professor Gerhard Kling, the central theme was not how to optimise better. It was something more unsettling:What if the thing we optimise for is the least stable input in the system?For data professionals, that question should immediately raise flags.The Real Problem: Expected Returns Are NoiseLet’s start with the uncomfortable truth.Expected returns are extremely difficult to forecast reliably.You can:Regress historical returnsUse factor modelsApply machine learningIncorporate macro signalsAnd yet, small changes in expected return estimates produce wildly different optimal portfolios.If your optimisation output changes dramatically because one expected return moved from 6% to 6.5%, your model is not robust. It is brittle.This is estimation error amplified by optimisation.In machine learning terms, it is the equivalent of a model that is hypersensitive to input noise.So, the workshop asked a practical question:If return forecasts are unstable, what happens if we stop optimising around them?Shift the Objective: Optimise Risk, Not ReturnHere is the key insight.While expected returns are noisy and unstable, covariance matrices tend to be more stable.Risk moves.Correlations shift.Volatility clusters.But those dynamics are still often more predictable than future returns.So instead of solving:Maximise expected return for a given level of riskWe shift to:Construct portfolios that manage risk structure intelligently.This is where risk-based portfolios enter.And this is where things get interesting for data professionals.Reading along? Catch the full workshop recap and technical walkthrough on our Packt Medium page.Data Science & ML Research Roundup🔵Provisioned Throughput (PT) on Vertex AI:Vertex AI’s Provisioned Throughput now guarantees reserved capacity and predictable performance for AI agents at scale. It supports 200+ models, including Anthropic and open models like Llama 4 and Qwen3, and powers multimodal workloads across Gemini, Veo, and Live API. New flexible terms, proactive scaling, and caching help teamsoptimizecost, latency, and growth.🔵Build unified intelligence with Amazon Bedrock AgentCore:As sales teams scale, toggling between Salesforce, support tickets, and Redshift slows customer intelligence. AWS solved this with CAKE, a customer-centric chat agent built on Amazon BedrockAgentCore. CAKE orchestrates Neptune, DynamoDB, OpenSearch, web search, and row-level security in parallel, delivering secure, context-rich insights in under 10 seconds through unified, multimodal enterprise data orchestration.🔵Gemini Enterprise Agent Ready (GEAR) program now available, a new path to building AI agents at scale:Agentic software is reshaping enterprise workflows, and Google’s Gemini Enterprise Agent Ready (GEAR) program is now open to all developers. As part of the Google Developer Program, GEAR offers 35 monthly learning credits, hands-on labs, and learning paths on agent architecture and ADK. Members can earn badges and pursue certifications to build production-ready, enterprise-grade AI agents.🔵Evaluating AI agents: Real-world lessons from building agentic systems at Amazon.As generative AI shifts from prompt-based apps to autonomous agentic systems, evaluation must evolve too. Amazon shares a comprehensive framework for assessing AI agents in production, combining an automated evaluation workflow with anAgentCoreevaluation library. The approach measures tool use, reasoning, memory, error recovery, and task success, enabling continuous monitoring, HITL auditing, and framework-agnostic performance insights.🔵Expanding Vertex AI with Claude Opus 4.6.Google Cloud has addedAnthropic’sClaude Opus 4.6 and Claude Sonnet 4.6 to Vertex AI, expanding its frontier model lineup. Opus 4.6 excels at enterprise workflows, coding, financial analysis, and complex agentic tasks, while Sonnet 4.6 balances intelligence and speed. With Agent Builder, Agent Engine, Provisioned Throughput, governance controls, and 1M-token context preview, Vertex AI delivers a full-stack platform for secure, scalable enterprise agents.🔵How CyberArk uses Apache Iceberg and Amazon Bedrock to deliver up to 4x support productivity:CyberArk cut case resolution time by up to 95% by combining Amazon Bedrock with Apache Iceberg. Using AI-generated grok patterns, Bedrock automates log parsing across diverse formats, whilePyIcebergenables single-stage, serverless table creation without crawlers. The result is zero-touch log ingestion, faster Athena queries, secure PII handling, and autonomous AI agents that perform root cause analysis in minutes, not days.🔵zai-org/GLM-5:GLM-5 is a 744B-parameter open-source model built for complex systems engineering and long-horizon agentic tasks. Itexpands onGLM-4.5 with more pretraining data, DeepSeek Sparse Attention for cost-efficient long context, and a new asynchronous RL framework called slime. GLM-5 deliversstate-of-the-artopen-source performance in reasoning, coding, and agentic benchmarks, narrowing the gap with frontier models.🔵MiniMaxAI/MiniMax-M2.5:MiniMax-M2.5 is a frontier modeloptimizedfor coding, agentic tool use, search, and professional office work. Trained with large-scale reinforcement learning across 200,000+ environments, it achieves 80.2% on SWE-Bench Verified and strongBrowseCompresults, while running 37% faster than M2.1. With 100 tokens per second throughput and ultra-low pricing, M2.5 delivers high-performance agents at a fraction of frontier model costs.🔵Qwen/Qwen3.5-397B-A17B:Qwen3.5-397B-A17B is a 397B-parameter multimodal foundation model with 17B active parameters, combining a vision encoder and causal language model. It features early fusion vision-language training, a hybrid GatedDeltaNetplus sparseMoEarchitecture, large-scale reinforcement learning, and support for 201 languages. The hosted Qwen3.5-Plus version adds 1M context, built-in tools, and production-ready capabilities.🔵nvidia/personaplex-7b-v1:NVIDIAPersonaPlexis a 7B-parameter real-time speech-to-speech conversational model built for full-duplex interaction. It simultaneously understands and generates streaming audio, enabling interruptions and natural turn-taking. Conditioned on voice and persona prompts, it controls vocal style and role behavior. Based on the Moshi architecture, it supports 24kHz audio and is commercially available under the NVIDIA Open Model License.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0

Merlyn from Packt
12 Dec 2025
Save for later

Are AI Agents Ready for Real Work Yet?

Merlyn from Packt
12 Dec 2025
What’s hype, what’s usable, and how data teams should prepare next.Subscribe|Submit a tip|Advertise with Us🧩 Welcome to DataPro 161. Our Expert Insight this week features Anas Riad, Data Scientist, BI Consultant, and ML Engineer at AdWay, as he maps what the next decade of AI could actually look like. From small language models (SLMs) and model routing to the rise of AI agents and the long-range pull of AGI, he breaks down what is moving from hype to real deployment, and what still needs serious guardrails around security, privacy, and energy costs. If you are building in AI, leading data teams, or trying to future-proof your career, this edition focuses on the decisions that will matter most between 2025 and 2035.We are also bringing something new to our readers, straight from experts with first-hand industry experience. Packt has launched live AMA sessions with industry leaders on the Packt DataML YouTube channel, hosted by our Growth Lead Abhishek Kaushik. These sessions give you direct access to practical guidance, career clarity, and real world insights from voices shaping the future of Python, AI, BI, and data science.In this edition, we have included Anas’s take on why smaller models will win more real-world workflows, how AI agents are evolving beyond demos, and what individuals and organizations should do now to stay productive without losing the human and ethical core of the work. You will find the extended highlights below.Knowledge Partner Spotlight: OutskillAt Packt, we’ve partnered withOutskillto help readers gain practical exposure to AI tools through free workshops, complementing the deeper, hands-on, expert-led experiences offered throughPackt Virtual Conferences.Learn AI tools, agents & automations in just 16 hours (End of Year offer)Outskill offers a focused 16-hour, two-day live AI program designed to help professionals enter 2026 more confident, faster, and AI-ready. As part of their Holiday Season Giveaway, the first 100 participants can join free and unlock bonus AI resources including a prompt library, monetization roadmap, and a personalized AI toolkit, making this a practical on-ramp for anyone serious about staying relevant in an AI-driven workplace.Register here for $0 (first 100 people only)💡 Workshop Spotlight: Applied Mathematics of Machine LearningJoin Tivadar Danka for an intensive, hands-on workshop on January 24, where you will build a rock-solid mathematical foundation for machine learning from the ground up. In this live session, you will go beyond model.fit() and implement core ML concepts from scratch in Python and NumPy, covering linear algebra, calculus, probability, and optimization through a full end-to-end linear regression workflow. Whether you are an aspiring data scientist, ML engineer, or Python developer transitioning into AI, this workshop connects theory to real implementation and helps you truly understand how modern ML systems work. Use EARLY50 for early access savings and reserve your seat.Register Now and Save With the Early Bird OfferUse EARLY50 for early access savings and reserve your seat.Cheers,Merlyn ShelleyGrowth Lead, PacktThe Future of Machine Learning and AIAgentic AI, AI Agents, and What’s Coming Next (2025–2035 Outlook)Artificial intelligence is no longer a future concept. It is already embedded in how we write, code, analyze data, design products, and make decisions. What has changed over the last few years is not the existence of AI, but its accessibility. Tools that were once reserved for large research labs and tech giants are now available on personal laptops, phones, and everyday workflows.In a recent episode ofPackt Talks, we sat down withAnas Riad, a data scientist, BI consultant, and ML engineer at AdWay, to unpack where AI and machine learning are heading next. The discussion spanned small language models, AI agents, industry disruption, ethics, energy costs, and the long-term question everyone is asking. Are we moving toward AGI, and what does that actually mean for society?This blog distills that conversation into a structured outlook on what the next decade of AI could look like, and how individuals and organizations should prepare.From Hype to Everyday UtilityMachine learning has existed for years, but only recently has it become part of everyday life. The real shift happened when AI became usable by non-experts. Today, anyone can interact with powerful models on their phone or laptop without understanding the mathematics behind them.This democratization explains the current hype cycle. AI feels revolutionary not because it is new, but because it is finally accessible. At the same time, the pace of progress is so fast that relevance feels temporary. What feels cutting-edge today can feel outdated in months.That speed is both exciting and destabilizing. It forces individuals and businesses to think less about mastering a single tool and more about staying adaptable.The Near-Term Shift: Smaller Models, Smarter DeploymentDespite the constant release of larger and more capable models, the next year is unlikely to be defined by dramatic breakthroughs. Instead, the focus is shifting towardefficiency.One of the most important trends is the rise ofsmall language models (SLMs). Not every task requires a massive, multi-billion parameter model. In fact, using large models for simple tasks is often slower, more expensive, and unnecessary.Small models excel when the task is narrow. Summarization, classification, lightweight reasoning, or structured extraction can often be done faster and cheaper with an SLM. Large models still matter for complex reasoning, multi-modal understanding, or long-context tasks, but the future is not one model doing everything.The real change is architectural. Systems will increasingly route tasks to the right model rather than defaulting to the largest one available. This improves speed, cost, and deployability, especially for local and edge use cases.What Changes for Users?From a user perspective, the difference between large and small models will mostly be invisible. What users will notice is faster responses, lower costs, and AI that feels more embedded into everyday tools rather than accessed through a single chat interface.The key shift is optimization. Instead of asking, “What is the best model?” teams will ask, “What is the right model for this task?” This mindset is essential for building scalable AI systems.Industry Impact: No Sector Is ImmuneAI is already reshaping software engineering, data science, and analytics. Code is written faster, debugging is assisted in real time, and deployment pipelines are increasingly automated. Tasks that once took days now take hours.Beyond tech, the impact is spreading everywhere:Healthcareis seeing early gains in diagnostics, scheduling optimization, and treatment modeling.Financeis using AI for credit risk, fraud detection, and portfolio optimization.Operations and logisticsare being optimized through predictive modeling and automation.Creative industriesare seeing massive productivity gains in writing, design, video, and music.The long-term implication is clear. AI adoption is no longer optional. Organizations that resist it will fall behind competitors who use it to move faster and operate more efficiently.Training, Architecture, and the Rise of AI AgentsOne of the most misunderstood aspects of modern AI is what it means to “use AI well.” It is not about chasing every new framework or model release. Success is measured by productivity gains, not by tool count.👉 Continue reading the full article on the Packt Medium handle.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0
Merlyn from Packt
04 Dec 2025
Save for later

Pablo Moreno on How Agentic AI Solves Broken Automation Workflows

Merlyn from Packt
04 Dec 2025
The gaps in automation and how agentic AI fills themUpgrade from Airflow 2 to 3 (without breaking your DAGs)Data teams at WeWork, Glassdoor, and Foursquare are already on Airflow 3. Are you?Join the Airflow experts at Astronomer on December 11 at 11am ET/4pm GMT to see what’s new and get tips on planning your upgrade:DAG versioning, human-in-the-loop, and more!Native support for MLOps + GenAI workflowsBest practices for upgrading your DAGs (without breaking them)Attend live for a chance to win a physical copy of the Practical Guide to Airflow 3!Save your spotSponsoredWelcome to DataPro Issue 160.This week, we’re bringing you a powerful edition packed with expert insights from across the AI landscape. In our latest Packt Talks session, Abhishek sits down with Pablo Moreno, AI Product Manager at Board, to break down one of the most urgent challenges facing data and automation teams today: why traditional workflows keep breaking, and how agentic AI is emerging as the scalable, adaptable solution enterprises actually need.In this issue, we dive into a structured, problem-solving walkthrough of the episode, covering real-world use cases from pharma and HR, the rise of Crew AI, new AI-powered roles, and what the shift from deterministic automation to autonomous agents really means for practitioners. If you're navigating automation, building AI systems, or leading data strategy, this one’s loaded with takeaways you can apply instantly.💡Workshop Spotlight:Algorithmic Trading with Python (Cohort 2)Only 10 seats left. Join Jason Strimpel on December 12 for a hands-on workshop where you’ll design, test, and deploy algorithmic trading strategies in Python with pandas, VectorBT, and the Interactive Brokers API. Whether you’re leveling up as a quant or sharpening your finance toolkit, this live session covers the full workflow from finding edges to executing real strategies. Choose a Full Pass or VIP Pass, use FINAL20 for 20 percent off, and secure your spot before it’s gone.Book Now and Save 20% – Only 10 Seats LeftCheers,Merlyn ShelleyGrowth Lead, PacktSubscribe|Submit a tip|Advertise with UsWhen Automation Breaks: How Agentic AI Solves Hidden Problems in Packt Talks with Pablo MorenoEvery company believes their automation is working, until it suddenly breaks. A small formatting change, a new policy variation, or a shift in user behavior can turn even the most expensive automated workflow into a bottleneck. This is the silent problem most businesses face: systems built to save time often create more manual work in the long run.In the latest episode ofPackt Talks, our GLAbhishek Kaushikdives into this challenge withPablo Moreno, AI Product Manager atBoard, a leader in financial planning and performance management software. Together, they unpack why traditional automation struggles to keep up with modern complexity and howagentic AIis rewriting the blueprint for scalable, resilient operations. The conversation sets the foundation for a deeper exploration of real-world cases, emerging frameworks, and the future of human-AI collaboration, insights that shape the rest of this article.The Automation Myth: Why Your Workflows Still BreakTraditional automation has helped companies save time for decades, but its biggest weakness remains unchanged: it can’t adapt. When business conditions shift, when documents look different, or when people ask slightly different questions, automated systems often fail. This mismatch between rigid workflows and fast-changing environments has widened over the last decade. Agentic AI steps in as a new paradigm, one where automation doesn’t just follow rules but makes decisions within defined boundaries.TakeawaysAutomation saves time but fails in dynamic, unpredictable scenarios.The pace of change in the last 5–10 years has outpaced deterministic workflows.Agentic AI thrives where traditional automation breaks, handling complexity without endless reprogramming.77% of enterprises are exploring agentic AI, with 30% already seeing major reductions in manual work.Problem in Focus: When Rule-Based Automation Turns into Technical DebtMany organizations rely on hyperautomation, systems built on stacks of rules, templates, and RPA bots. But every new rule becomes a maintenance cost. A single formatting change can break an automated pipeline. In the pharma industry example Pablo Moreno shares, scanning marketing documents for legal disclaimers turned into an expensive cycle of bot retraining.TakeawaysRule-heavy automation becomes brittle and expensive at scale.Minor variations (orientation, typos, layout changes) often break RPA workflows.Over-reliance on deterministic rules leads to high maintenance debt.The real cost of automation isn’t deployment — it’s constant upkeep.How Agentic AI Fixes the RPA ProblemAgentic AI doesn’t rely on rigid patterns. It understands context, recognizes variations, and adapts to change. In the pharma compliance case, agents could detect legal phrases regardless of spelling errors, text rotation, or synonyms. Instead of adding rules, the system learned patterns. This shift replaced hours of manual review and repetitive bot tuning.TakeawaysAgentic AI extracts meaning, not formatting, reducing the need for rules.Agents adapt to document variance with little intervention.Maintenance drops because the system generalizes from examples.Compliance workflows become faster, cheaper, and more reliable.Why Crew AI Works: Solving the Complexity ProblemChoosing the right framework matters. Pablo highlights Crew AI as his preferred approach because it enables multi-agent processes while retaining simplicity. Unlike low-code/no-code tools that struggle once workflows become nonlinear or multi-context, Crew AI supports scalable architectures, evolving use cases, and quick iteration cycles.TakeawaysCrew AI enables multi-agent workflows without heavy engineering overhead.Low-code/no-code tools are fast but tend to break once complexity grows.Crew AI supports new agents, tools, integrations, and iteration loops.Scalability and adaptability should guide platform decisions, not speed alone.The HR Chatbot Challenge: When Reality Is MessyA multinational pharma company attempted to build an HR chatbot for 100,000 employees across 40 countries. The complexity was enormous: policies varied by 20–30 percent, acronyms differed across teams, and employee questions were full of local nuance. The breakthrough came from building a robust knowledge base, including an acronym dictionary, and extensively training the system to recognize contextual differences.TakeawaysReal-world enterprise AI must handle messy, inconsistent data.Local nuances in HR policies can’t be hard-coded into rules.Knowledge bases and acronym dictionaries significantly improve AI accuracy.Good framing and good inputs are critical for agentic AI performance.Continue reading the full article on the Packt Medium handle.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0

Merlyn from Packt
02 Dec 2025
Save for later

What Walmart’s Senior Data Scientist says about skills, hiring, and staying relevant.

Merlyn from Packt
02 Dec 2025
Why product DS, applied DS, and AI engineering roles are exploding right now.Subscribe|Submit a tip|Advertise with Us🧩Welcome to DataPro 159. Our Expert Insight this week features Walmart Senior Data Scientist Karun as he breaks down how AI, automation, and GenAI are rewriting the rules of data science. Whether you are job hunting or leveling up your skills, his advice cuts straight to what matters. We are also bringing something new to our readers, straight from experts with first-hand industry experience. Packt has launched live AMA sessions with industry leaders on the Packt DataML YouTube channel, hosted by our Growth Lead Abhishek Kaushik. These sessions give you direct access to practical guidance, career clarity, and real world insights from voices shaping the future of Python, AI, and data science.In this edition, we have included Karun’s take on evolving data science roles, portfolio strategies that stand out, and the skills hiring managers value most in 2025 and 2026. You will find the extended highlights below.Knowledge Partner Spotlight: OutskillAt Packt, we’ve partnered withOutskillto help readers gain practical exposure to AI tools through free workshops, complementing the deeper, hands-on, expert-led experiences offered throughPackt Virtual Conferences.Still doing manual tasks repetitively? Learn AI to automate 50% of your work.Outskill is hosting a 2-Day Live AI Mastermind, an intensive training on AI tools, automations, and agent building designed to make you indispensable. As part of their Holiday Season Giveaway, the first 100 participants can join for free, even though the program normally costs $395. The live sessions run Saturday and Sunday from 10 AM to 7 PM EST, and attendees unlock over $5000 worth of AI resources, including a Prompt Bible, a roadmap to monetizing with AI, and a personalized AI toolkit builder.Register here for $0 (first 100 people only)💡 Workshop Spotlight: Algorithmic Trading with Python (Cohort 2)Join Jason Strimpel for a hands-on workshop on December 12, where you’ll design, test, and deploy algorithmic trading strategies in Python using pandas, VectorBT, and the Interactive Brokers API. Whether you’re a quant-in-training or a finance pro, this live session takes you through the entire workflow, from identifying trading edges to executing real-world strategies. Pick a Full Pass or VIP Pass, use FINAL20 for 20% off, and reserve your seat.Register Now and Save With the Early Bird OfferCheers,Merlyn ShelleyGrowth Lead, PacktThe Real Story Behind Today’s Data Science Job Market and How to Stand Out with Karun ThankachanIf you feel like the data science world is shifting under your feet, you are not alone. In the latest episode of Pack Talks, Abhishek Kaushik sat down with Karun, a senior data scientist at Walmart, to unpack what it really takes to break into and thrive in this field today. The conversation was honest, pragmatic, and sprinkled with hard-won wisdom from someone who has navigated one of the toughest job markets in recent history.This article brings you the essential takeaways: how the role of a data scientist is evolving, what skills truly matter now, how to build a competitive portfolio, and the strategies that actually work for landing interviews and surviving them.Let’s dive in.From Pandemic Turbulence to Walmart: Karun’s Nonlinear JourneyKarun’s story begins like many hopeful data scientists. He moved to the US for his master’s at Carnegie Mellon, rode the initial excitement of landing an internship, then watched that offer disappear during the COVID downturn. It was a moment when thousands of fresh graduates questioned whether the field still had space for them.Instead of giving up, he leaned into referrals, rebuilt his pipeline, and secured a data science internship at Amazon. That later became a full-time role, and today he works on high-impact data science initiatives inside Walmart.His journey is a reminder that resilience still matters. The field is changing, but it is far from closing its doors.Is Data Science Still a Good Career? The Market Has Evolved, Not ShrunkWhen asked the million-dollar question, Karun was clear. The career is still viable, but the expectations have shifted.Here is how the field has changed since 2017.1. Model building has become easierThe rise of tools like XGBoost and transformers reduced the complexity of many modeling tasks. You no longer need niche mathematical expertise to build a model that works.2. Business impact has become the differentiatorHiring managers want people who can translate problems into solutions that move metrics, reduce costs, or improve customer experience.3. Entry-level roles now demand mid-level thinkingThanks to automation and AI assistants, junior roles expect stronger system design, problem framing, and measurable impact.4. Career growth now mirrors software engineeringInstead of staying at the “modeling” layer, data scientists are expected to grow into architecture, experimentation strategy, and problem definition.The bottom line: AI did not remove data science jobs. It elevated the bar for what a data scientist does.The Four Data Science Roles You Need to Understand in 2025Karun breaks down today’s landscape into four categories. Knowing the difference helps you target your learning, resume, and projects.Role TypeWhat You Actually DoHow AI Affects ItProduct Data ScientistA/B testing, causal inference, experimentationStill high in demand. Requires sound statistical reasoning.Applied Data ScientistBuild customer-facing models and deploy pipelinesCoding and model tuning remain essential. Not heavily automated yet.Research Data ScientistCreate new model architectures and algorithmsLeast affected by automation. Requires deep ML expertise.AI Engineeroptimize LLM pipelines, work with vector DBs, prompts, deployment. Growing rapidly as companies scale GenAI.Inside Walmart, Karun shared that they already integrate ChatGPT-powered workflows internally. But even with AI tools in the loop, human oversight, system thinking, and engineering maturity remain essential.Python vs R in 2025: The Debate Is OverKarun did not hesitate here.Python has won.Why:It integrates with big data tools like PySparkIt is production friendlyIt is the default language in ML teamsIt has unmatched community supportR is still relevant for specialized statistical work, but new entrants are better off starting with Python. For performance heavy systems, C and C++ still matter, especially for latency sensitive pipelines.Want Recruiters To Notice You? Build Business Oriented ProjectsKarun’s portfolio advice is refreshingly practical.Stop doing random datasets. Start doing business problems.He recommends:Kaggle competitions with prize moneyRetail and financial datasets like M5 forecasting, H&M recommendations, Jane Street market predictionProjects where you quantify your performance against leadersOne golden rule:Always show how close you are to rank one. Not just your accuracy.👉 Continue reading the full article on the Packt Medium handle.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0

Merlyn from Packt
19 Nov 2025
Save for later

New Insights From Our Live AMA With Juan Gabriel Salas

Merlyn from Packt
19 Nov 2025
Python insights plus a chance to build real trading strategies.Subscribe|Submit a tip|Advertise with Us🧩Welcome to DataPro 158This week we are bringing something new to our readers. We have launched live AMA sessions with industry experts on the Packt DataML YouTube channel, hosted by our Growth Lead Abhishek Kaushik. These sessions give you direct access to practical guidance, career clarity, and real world insights from voices shaping the future of Python, AI, and data science. In this edition, we highlight takeaways from our latest conversation with Juan Gabriel Salas, one of Udemy’s highest rated Python and data science instructors. Juan shared actionable advice on learning Python with purpose, bridging the gap between syntax and real workflows, developing strong data intuition, and building portfolios that employers care about. You’ll find the recap below.Knowledge Partner Spotlight: OutskillAt Packt, we’ve partnered withOutskillto help readers gain practical exposure to AI tools through free workshops, complementing the deeper, hands-on, expert-led experiences offered throughPackt Virtual Conferences.Become an AI Genius in One WeekendAs part of their Black Friday offer, Outskill is opening access to their 2 day LIVE AI Mastermind for free. This 16 hour intensive program covers AI tools, automations, and agent building, and is rated 9.8 on Trustpilot. It is usually priced at $395, but DataPro readers can join at no cost while seats last. The live sessions run Saturday and Sunday from 10 AM EST to 7 PM EST.Save your spot now, limited seats available.💡 Workshop Spotlight: Algorithmic Trading with Python (Cohort 2)Join Jason Strimpel for a hands-on workshop on December 12, where you’ll design, test, and deploy algorithmic trading strategies in Python using pandas, VectorBT, and the Interactive Brokers API. Whether you’re a quant-in-training or a finance pro, this live session takes you through the entire workflow, from identifying trading edges to executing real-world strategies.Pick a Full Pass or VIP Pass, use EARLY20 for 20% off, and reserve your seat.Register Now and Save With the Early Bird OfferCheers,Merlyn ShelleyGrowth Lead, PacktMastering Python and Data Science: Insights From a Live AMA WithJuan Gabriel SalasWith an aim to improve the reader experience, we are rolling out a new AMA series in partnership with our Expert Network to bring firsthand insights directly from industry leaders in the data world.Abhishek, our Growth Lead for the data science vertical, is hosting these live AMA sessions on thePackt DataML YouTube channelto help learners gain clarity, confidence, and direction in their data journey.This week’s session featuresJuan Gabriel Salas, a top rated Udemy instructor and founder of Frogames Formación. Juan is known for his practical teaching style, clear explanations, and deep experience helping thousands of learners master Python and build data science careers. Over the course of an hour, he shared a wide range of actionable insights on how to learn Python effectively, build strong intuition, create impactful portfolios, and navigate the fast evolving world of data science and machine learning.This article brings together all those insights in a single narrative so that every data learner can benefit from the clarity and wisdom that came out of the session.The Big Themes of the SessionPython fundamentals. Data workflows. Portfolio strategy. Math intuition. Future skills. And everything in between.To make this recap easier to navigate, we have organized the insights into six major themes that emerged during the AMA.1. Where Should Beginners Start: Python or Data Science?The conversation kicked off with a classic dilemma. Should learners start with Python basics or jump straight into data science?Juan’s advice was simple and rooted in experience.Start with Python, but not in a way that traps you in endless syntax drills. Instead, learn Python with a purpose. Focus on the parts of the language that matter for data work.You do not need to become a software engineer before becoming a data practitioner. But you do need enough fluency to manipulate data, write reusable code, and understand how your tools work behind the scenes.Python is not just another skill. It is the language that holds your entire workflow together.What matters most at the start:Knowing how to work with lists and dictionariesReading and writing filesUnderstanding functions and basic logicGetting comfortable with problem solvingOnce these fundamentals feel natural, transitioning into data science becomes far more enjoyable.2. Closing the Gap Between Syntax and Real Data Science WorkMany learners understand Python basics but freeze when facing a blank project. They know what a loop is but not how to apply one in a data pipeline. They understand lists but not how to use them in a transformation.Juan explained that this is not a lack of skill. It is a mindset shift that has not yet fully developed.Data science is less about writing isolated lines of code and more about seeing the bigger picture. It requires thinking insystems and workflowsrather than steps.You move from:“How do I write this line of code?”to“How do I design a process that answers this question?”You start seeing your work as a sequence of stages:Load. Clean. Explore. Transform. Model. Evaluate. Communicate.This shift is what turns a coder into a data practitioner.3. How to Approach a Real Data Science ProblemOne of the most valuable parts of the AMA was Juan’s breakdown of what to do when facing a new data science project from scratch. Instead of jumping into libraries or algorithms, he recommended focusing on three checkpoints that define successful work.Checkpoint 1: Understand Data QualityBefore choosing a model or writing transformations, you must examine the data. Learn to recognise missing values, duplicated rows, strange outliers, or mislabeled categories. This is where the bulk of practical skill is formed.Checkpoint 2: Choose the Right Modeling ApproachIs this a classification problem? A regression? A time series forecast? A clustering experiment?Knowing what problem you are solving is more important than the model itself.Checkpoint 3: Define Evaluation Metrics EarlyAccuracy is not always the right metric. Sometimes you need precision, recall, AUC, F1, RMSE, or business specific KPIs.Thinking about metrics early forces clarity.Together, these checkpoints create a stable foundation for any analysis.4. How Tiny Python Mistakes Can Break Entire ML WorkflowsJuan shared a humorous yet painful story about how a single indexing mistake caused inconsistencies across an entire machine learning pipeline. The model trained successfully. The predictions looked plausible. But the results made no sense.The culprit was a tiny transform that shifted rows.This resonated deeply with the audience because it illustrates a truth that every data practitioner eventually learns.Machine learning magnifies small mistakes.One misaligned array. One swapped column. One incorrect filter. One wrong assumption during data cleaning.This is why debugging is one of the most underrated data science skills. Being patient, methodical, and curious makes you far more effective than knowing every library function.5. What Makes a Portfolio Stand Out to EmployersPerhaps the most practical part of the AMA was Juan’s guidance on portfolio building. Many learners build beautiful notebooks that fail to impress employers simply because they focus on the wrong things.Juan broke down the three elements that make a portfolio truly meaningful.1. Use Real World DataAvoid overused toy datasets. Instead, explore domains you care about. Scrape data. Gather logs. Use messy CSVs. These reveal how you work, not how well you follow tutorials.2. Show Your ThinkingEmployers want to know why you chose an approach, what decisions you made, and how you debugged problems. Narrative clarity matters just as much as code.3. Focus on Depth, Not QuantityFive shallow projects will not beat two strong, thoughtful, well documented projects with clear reasoning and solid results.A strong portfolio is a story, not a showcase.👉 Continue reading the full article on the Packt Medium handle.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0
Merlyn from Packt
13 Nov 2025
Save for later

How AI Agents Think (and Why It Matters)

Merlyn from Packt
13 Nov 2025
Limited-time Humble Bundle: 3 best-selling Packt titles for $18 — learn, grow, and give back.Subscribe|Submit a tip|Advertise with Us🧩 Welcome to DataPro #157Your weekly pulse on data, AI, and the people shaping its future.This week we feature Sagar Lad, a Data & AI Solution Architect who’s passionate about making complex AI ideas approachable. In his latest article, “What Are AI Agents? A Simple Guide,” Sagar unpacks one of the most fascinating developments in AI, systems that can perceive, reason, and act autonomously.From self-driving cars that interpret their surroundings to chatbots that learn from conversations, Sagar explains how AI agents work under the hood, detailing how they perceive input, make decisions, and evolve through feedback. It’s a crisp, example-driven read that helps you grasp how machine autonomy is reshaping everything from robotics to analytics.📘 Deal of the Week: Packt’s Humble BundleCalling all data-driven professionals! Strengthen your Python and BI foundations with 3 best-selling Packt books for just $18. This curated bundle includes must-reads like Python Machine Learning by Example, Causal Inference and Discovery in Python, and Modern Time Series Forecasting with Python. Learn from experts, level up your analytics skills, and support Direct Relief with your purchase. Over 2,600 bundles sold and $4,500 raised for charity so far, don’t miss your chance to grab 25 essential data books for a fraction of the price.💡 Workshop Spotlight: Algorithmic Trading with Python (Cohort 2)Join Jason Strimpel for a hands-on workshop on December 12, where you’ll design, test, and deploy algorithmic trading strategies in Python using pandas, VectorBT, and the Interactive Brokers API. Whether you’re a quant-in-training or a finance pro, this live session takes you through the entire workflow, from identifying trading edges to executing real-world strategies.Let’s dive in 👇Cheers,Merlyn ShelleyGrowth Lead, PacktWhat Are AI Agents? A Simple Guide by Sagar LadArtificial Intelligence (AI) is transforming the way we live and work. One of the most exciting developments in this field is the rise ofAI agents. But what exactly are they? Let’s break it down in a way that’s easy to understand.What Is an AI Agent?AnAI agentis a computer program that canperceive its environment,make decisions, andtake actionsto achieve a goal. Think of it like a digital assistant that doesn’t just follow instructions — it learns, adapts, and acts intelligently.Task Execution Flow by AI AgentsExample:Imagine a robot vacuum cleaner. It senses where dirt is, avoids obstacles, and decides the best path to clean your room. That’s an AI agent in action!Internal Working of an AI Agent1. Perception (Input Layer)The agent receives data from its environment.This could be sensor data (for robots), user input (for chatbots), or digital signals (for trading bots).Internally, this data is processed into a format the agent can understand often as structured data or features.Example:A self-driving car uses cameras and sensors to detect pedestrians, traffic lights, and road signs.2. Knowledge Base / State RepresentationThe agent maintains a model of the world or its environment.This could be a simple memory of past actions or a complex map of the surroundings.It helps the agent understand context and make informed decisions.Example:A chess-playing agent remembers the current board state and possible moves.3. Decision-Making (Reasoning Engine)Based on the perceived data and internal state, the agent decides what to do next.This can involve:Rule-based logic(if-then rules)Search algorithms(finding the best path or solution)Machine learning models(predicting outcomes)Reinforcement learning(learning from trial and error)Example:A recommendation agent uses a machine learning model to suggest movies based on your viewing history.4. Action (Output Layer)The agent performs an action in the environment.This could be sending a message, moving a robot arm, or updating a database.Example:A chatbot replies to a user query with a helpful answer.5. LearningSome agents improve over time by learning from feedback or new data.Supervised learning(learning from labeled data)Unsupervised learning(finding patterns)Reinforcement learning(learning from rewards and penalties)Example:A game-playing agent gets better by playing thousands of games and learning which strategies win.🔄 The Agent LoopHere’s a simplified loop that most AI agents follow:Perceive → Analyze → Decide → Act → Learn (optional) → Repeat👉 Continue reading the full article on the Packt Medium handle.📘 Why You Shouldn’t Miss This Humble BundleIf you’re looking to strengthen your data and analytics skills, this is an opportunity worth taking. Packt’s Modern Data Analytics & Business Intelligence Mastery Bundle brings together 25 carefully selected titles from trusted experts in data, analytics, and AI, and you can start with 3 bestsellers for just $18.Inside, you’ll find practical guides like Python Machine Learning by Example, Causal Inference and Discovery in Python, and Modern Time Series Forecasting with Python. These books go beyond theory and help you apply what you learn to real-world challenges. Whether you’re building dashboards, experimenting with predictive models, or mastering Power BI and Microsoft Fabric, this bundle provides the hands-on knowledge you need to stay ahead in an evolving field.Over 2,600 bundles sold and $4,500 raised for charity so far shows the impact this collection is already making. When you buy, you’re not only investing in your growth but also supporting Direct Relief.It’s a valuable way to learn, grow, and give back, available for a limited time.👉 Grab your bundle before it’s gone!Grow your skills while supporting a great causeSee you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0

Merlyn from Packt
04 Nov 2025
Save for later

What’s powering AI’s next leap? LongCat Flash Omni, DeepAgent, SkyRL & more

Merlyn from Packt
04 Nov 2025
From multimodal LLMs to self-thinking agents, see what’s driving AI’s next leap.👋 Hello ,Welcome to DataPro #155 ➖Where Models Get Smarter, Agents Get Autonomous, and AI Gets Real-Time.This week’s edition explores the frontier of intelligent systems that see, reason, and act. From Meituan’s LongCat Flash Omni and DeepAgent’s unified reasoning to OpenAI’s gpt-oss-safeguard and SkyRL tx, AI is rapidly evolving toward autonomy, speed, and safety. We also look at how multimodal RAG, ethical AI, and data mesh are redefining how we build and scale intelligence.Knowledge Partner Spotlight: OutskillAt Packt, we’ve partnered with Outskill to help readers gain practical exposure to AI tools through free workshops, complementing the deeper, hands-on, expert-led experiences offered through Packt Virtual Conferences.If you're interested in enhancing your AI skills, Outskill’s LIVE 2-Day AI Mastermind offers a 16-hour training on AI tools, automations, and agent-building. This weekend’s sessions (Saturday and Sunday, 10 AM–7 PM EST) are available at no cost as part of their Black Friday Sale, providing a great opportunity to elevate your knowledge in just two days.Learn AI tools, agents & automations in just 16 hoursJoin now, limited free seats available!This week’s highlights:🔸LongCat Flash Omni:Meituan’s open 560B multimodal model for real-time interaction🔸DeepAgent: A unified reasoning agent that thinks, searches, and acts autonomously🔸SkyRL tx v0.1.0: Tinker-style reinforcement learning engine for local clusters🔸OpenAI gpt-oss-safeguard: Policy-conditioned safety reasoning models, open-weight and Apache 2.0🔸Does AI Need to Be Conscious to Care? Exploring the philosophy of artificial moral concern🔸Building Multimodal RAG: How to make retrieval truly visual and contextual🔸Covestro x Amazon DataZone: A blueprint for scaling data governance through data meshEach story in this issue unpacks a new layer in how AI learns, governs, and grows—so grab a coffee, settle in, and let’s dive into the full roundup.Cheers,Merlyn ShelleyGrowth Lead, PacktSponsored:🔸82% of data breaches happen in the cloud. Join Rubrik’s Cloud Resilience Summit to learn how to recover faster and keep your business running strong. [Save Your Spot]🔸Build your next app on HubSpot’s all-new Developer Platform,the flexible, AI-ready foundation to create, extend, and scale your integrations with confidence. [Start Building Today]Subscribe|Submit a tip|Advertise with UsTop Tools Driving New Research 🔧📊🔶 LongCat-Flash-Omni: A SOTA Open-Source Omni-Modal Model with 560B Parameters with 27B activated, Excelling at Real-Time Audio-Visual Interaction. Meituan’s LongCat Flash Omni is a 560B-parameter open-source multimodal model that activates 27B per token using shortcut-connected MoE. It extends text LLMs to vision, video, and audio with 128K context and real-time streaming through 1-second audio-visual interleaving at 2 fps duration-conditioned sampling. With modality-decoupled parallelism, it retains 90% text-only throughput and scores 61.4 on OmniBench, 78.2 on VideoMME, and 88.7 on VoiceBench, nearing GPT-4o performance.🔶 DeepAgent: A Deep Reasoning AI Agent that Performs Autonomous Thinking, Tool Discovery, and Action Execution within a Single Reasoning Process. Most agent frameworks still follow a fixed Reason–Act–Observe loop, but DeepAgent from Renmin University and Xiaohongshu redefines this with end-to-end deep reasoning. Built on a 32B QwQ backbone, it unifies thought, tool search, tool call, and memory folding within one stream. It dynamically retrieves tools from 16K+ APIs, compresses long histories into structured memories, and trains via Tool Policy Optimization (ToolPO) for precise tool use. DeepAgent achieves 69.0 on ToolBench and 91.8% success on ALFWorld, outperforming ReAct-style workflows in both labeled and open tool settings.🔶 Anyscale and NovaSky Team Releases SkyRL tx v0.1.0: Bringing Tinker Compatible Reinforcement Learning RL Engine To Local GPU Clusters. Anyscale and UC Berkeley’s NovaSky team released SkyRL tx v0.1.0, a local, Tinker-compatible engine that unifies training and inference for LLM reinforcement learning. It implements Tinker’s low-level API (forward_backward, optim_step, sample, save_state) and runs on user infrastructure. The update adds end-to-end RL, jitted sharded sampling, LoRA adapter support, gradient checkpointing, micro batching, and Postgres integration, enabling full RL training on 8×H100 GPUs with Tinker-level efficiency and open deployment.🔶 OpenAI Releases Research Preview of 'gpt-oss-safeguard': Two Open-Weight Reasoning Models for Safety Classification Tasks. OpenAI released gpt-oss-safeguard, two open-weight safety reasoning models, 120B and 20B parameters, that let developers enforce custom safety policies at inference time. Fine-tuned from gpt-oss and Apache 2.0 licensed, they replicate OpenAI’s internal Safety Reasoner used in GPT-5 and Sora 2. The models reason step by step on developer-supplied policies, outperform gpt-5-thinking on multi-policy accuracy, and fit on single-GPU setups for real moderation pipelines.Topics Catching Fire in Data Circles 🔥💬🔶 Does AI Need to Be Conscious to Care? This philosophical study explores that question through a precise framework. It distinguishes functional, experiential, and moral caring, showing that caring behaviors can exist without consciousness, as seen in bacteria, plants, and immune systems. While current AI systems display goal-directed, welfare-promoting behavior, they lack genuine concern. Consciousness-based and agency-based routes could both lead to artificial moral concern, suggesting caring exists on a spectrum. Future AI may combine conscious experience with robust agency, raising urgent ethical questions about artificial moral significance.🔶 Building a Multimodal RAG That Responds with Text, Images, and Tables from Sources. Retrieval-Augmented Generation (RAG) has long powered text-based chatbots, but extending it to images, tables, and graphs is far harder. Real documents, like research papers and corporate reports, mix text, formulas, and figures without consistent formatting, breaking the link between visuals and context. To fix this, a new multimodal RAG pipeline introduces context-aware image summaries using nearby text instead of isolated captions, and text-response-guided image selection, where visuals are chosen after the textual answer is generated. Together, these steps yield consistent, contextually grounded multimodal retrieval across complex documents.🔶 From Classical Models to AI: Forecasting Humidity for Energy and Water Efficiency in Data Centers. This blog explores how accurate humidity forecasting can improve the efficiency, reliability, and sustainability of AI data centers. It explains how temperature and humidity directly affect cooling systems, energy use, and water consumption, and presents a real-world case study using Delhi’s climate data. The post compares forecasting methods, AutoARIMA, Prophet, XGBoost, and deep learning, with prediction intervals to assess accuracy and uncertainty, aiming to identify the best tools for operational planning and environmental optimization in large-scale AI infrastructure.🔶 Scaling data governance with Amazon DataZone: Covestro success story. This blog explores how Covestro Deutschland AG reengineered its global data architecture by transitioning from a centralized data lake to a domain-driven data mesh using Amazon DataZone and the AWS Serverless Data Lake Framework (SDLF). The transformation empowered teams to manage data products independently while maintaining consistent governance, improving data sharing and visibility. Through AWS Glue, S3, and automated data quality checks, Covestro now operates over 1,000 standardized data pipelines, achieving faster delivery, stronger governance, and scalable analytics across the enterprise.New Case Studies from the Tech Titans 🚀💡🔶 How to design conversational AI agents? This blog explores how conversational AI is transforming the online shopping experience by replacing rigid keyword-based search with natural, intuitive interactions. It outlines seven key design principles for creating AI shopping agents that understand user intent, personalize recommendations, support multimodal input, and present rich visuals. The post also highlights best practices for building user trust, handling ambiguity gracefully, and leveraging Google Cloud’s Conversational Commerce tools and Figma’s component library to design adaptable, on-brand, and intelligent shopping experiences.🔶 How 5 agencies created an impossible ad with Gemini 2.5 Pro? Generative AI is rewriting the rules of creativity. With Gemini 2.5 Pro and Google’s suite of generative media models, Imagen, Veo, Lyria, and Chirp, brands are moving beyond traditional campaigns to design what was once impossible. From Slice’s AI-powered retro radio station and Virgin Voyages’ personalized “postcards from your future self,” to Smirnoff’s interactive party co-host and Moncler’s cinematic AI film, these projects show how imagination and technology now merge to create entirely new forms of storytelling and brand expression.🔶 Build intelligent ETL pipelines using AWS Model Context Protocol and Amazon Q: Building and maintaining ETL pipelines has long been one of the most time-consuming parts of data engineering. With conversational AI and Model Context Protocol (MCP) servers, teams can now automate much of that process, turning complex scripting into guided, natural language interactions. By integrating with AWS services like Redshift, S3 Tables, and Glue, organizations can generate, test, and deploy pipelines faster while preserving security and governance standards. This post demonstrates how data scientists and engineers can use conversational AI to extract data, validate quality, and automate end-to-end migrations from Redshift to S3, reducing manual effort, improving accuracy, and accelerating insight generation.🔶 Amazon Kinesis Data Streams launches On-demand Advantage for instant throughput increases and streaming at scale: Managing real-time data streams just became simpler and more cost-efficient with the launch of Amazon Kinesis Data Streams On-demand Advantage mode. This new capability introduces warm throughput for instant scalability during traffic spikes and a committed-usage pricing model that significantly lowers costs for steady, high-volume workloads. Designed for use cases ingesting at least 10 MiB/s or operating hundreds of streams per region, it eliminates the need to manually switch between capacity modes. The post explains how On-demand Advantage helps organizations handle predictable surges, optimize costs, and configure warm throughput up to 10 GiB/s, along with setup steps, pricing details, and best practices for maintaining high-performance streaming pipelines.Blog Pulse: What’s Moving Minds 🧠✨🔶 The Pearson Correlation Coefficient, Explained Simply: Understanding how variables move together is the foundation of predictive modeling. In this walkthrough, we explore how to calculate and interpret the Pearson correlation coefficient, a key step before fitting a regression model. Using a simple salary dataset with Years of Experience and Salary, the post explains how to visualize relationships with scatter plots, compute variance, covariance, and standard deviation, and finally derive the correlation coefficient. With a result of r = 0.9265, the example shows a strong positive linear relationship, confirming that simple linear regression is well suited for predicting salary based on experience.🔶 Graph RAG vs SQL RAG: Comparing how large language models reason over structured and connected data reveals valuable insights into retrieval-augmented systems. In this experiment, a Formula 1 results dataset was stored in both a SQL and a graph database, then queried using retrieval-augmented generation (RAG) with models like GPT-3.5, GPT-4, and GPT-5. Each model translated natural language into SQL or graph queries to answer questions about drivers, races, and championships. The results show that newer models like GPT-5 achieved near-perfect accuracy across both databases, while simpler models struggled more with graph data. The study concludes that RAG-equipped LLMs can reason reliably over either database type, letting teams choose whichever structure best fits their data without sacrificing performance.🔶 RF-DETR Under the Hood: The Insights of a Real-Time Transformer Detection. Object detection has come a long way from rigid anchor grids to adaptive Transformer architectures. RF-DETR, Roboflow’s latest real-time detection model, embodies that evolution. Building on DETR’s end-to-end design, Deformable DETR’s adaptive attention, and LW-DETR’s lightweight efficiency, RF-DETR fuses these innovations with a DINOv2 self-supervised backbone for domain adaptability and speed. The result is a model that achieves real-time performance without sacrificing accuracy, capable of both bounding box detection and segmentation. In essence, RF-DETR showcases how adaptive attention and self-supervised vision have made Transformers fast, flexible, and production-ready for modern computer vision tasks.🔶 Building secure Amazon ElastiCache for Valkey deployments with Terraform. Managing infrastructure through code is becoming essential for secure, scalable cloud deployments. Using Infrastructure as Code (IaC) with Terraform, this guide walks through building a secure Amazon ElastiCache for Valkey cluster, covering both serverless and node-based options. It demonstrates how IaC ensures consistent configurations for encryption, authentication, and network isolation across environments. The walkthrough details step-by-step deployment, from provisioning private subnets and KMS-encrypted storage to implementing token-based authentication and CloudWatch logging. The result is a reproducible, production-grade ElastiCache setup that combines automation, security, and cost efficiency through a modern Terraform workflow.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}
Read more
  • 0
  • 0
Success Subscribed successfully to !
You’ll receive email updates to every time we publish our newsletters.
Modal Close icon
Modal Close icon