BIPro

AI Assistant with Amazon Q and S3 clickable URLs, Value of Thoughtful Dashboard Design in TableauBecome an AI Generalist that makes $100K (in 16 hours)One of the biggest IT giants, TCS laid off 12,000 people this week. And this is just the beginning of the blood bath. In the coming days you’ll see not thousands, but millions of more layoffs & displacement of jobs. So what should you do right now to avoid getting affected? Invest your time in learning about AI. The tools, the use cases, the workflows – as much as you can.Join the World’s First 16-Hour LIVE AI Upskilling Sprint for professionals, founders, consultants & business owners like you. Register Now (Only 500 free seats)Date: Saturday and Sunday, 10 AM - 7 PM.Rated 4.9/5 by global learners – this will truly make you an AI Generalist that can build, solve & work on anything with AI.In just 16 hours & 5 sessions, you will:✅ Learn how AI really works by learning 10+ AI tools, LLM models and their practical use cases.✅ Learn to build and ship products faster, in days instead of months✅ Build AI Agents that handle your repetitive work and free up 20+ hours weekly✅ Create professional images and videos for your business, social media, and marketing campaigns.✅ Turn these AI skills into10$k income by consulting or starting your own AI services business.All by global experts from companies like Amazon, Microsoft, SamurAI and more. And it’s ALL. FOR. FREE. 🤯 🚀$5100+ worth of AI tools across 2 days — Day 1: 3000+ Prompt Bible, Day 2: Roadmap to make $10K/month with AI, additional bonus: Your Personal AI Toolkit Builder.Register Now (Only 500 free seats)SponsoredSubscribe|Submit a tip|Advertise with usWelcome to BIPro #108 -Where Better Dashboards BeginThis week, we’re going beyond dashboards as decoration and diving into dashboards that deliver. If you've ever opened a report and wondered, "What am I looking at, and why does it matter?", you’re not alone. In our lead story, we explore the value of thoughtful dashboard design in Tableau, unpacking four powerful design strategies, from guided analysis to executive scorecards, that can help you communicate insights with clarity and intent. It’s not just about data on a page; it’s about shaping stories your stakeholders can act on.Once you're primed on dashboard design, stick around, this issue is packed with technical deep dives and timely updates:🔹A practical Power BI workaround for start values in waterfall charts🔹A how-to on building an AI assistant with Amazon Q and S3 clickable URLs🔹What data literacy really means in 2025 (spoiler: it’s not what you think)🔹New Google Cloud agents and AI-native foundations for data teams🔹The debut of Spanner’s columnar engine, finally bridging OLTP and OLAP🔹And news you need to know: Power BI’s Copilot is going full-screen by default this SeptemberWherever you sit, BI analyst, data scientist, exec stakeholder, this edition offers tools and thinking to help you work smarter with data, not just harder.Let’s get into it.Sponsored👉 Join Snyk’s Sonya Moisset on August 28 at 11:00AM ET to explore how to secure AI-powered development from code to deployment. Learn how to protect your SDLC, mitigate risks in vibe coding, and earn 1 CPE credit. Register today!👉 Webinar alert! Mobile experts from Bitrise and Embrace break down advanced CI/CD tips and real-user insights to help you speed up builds & deliver top-quality apps. Register here.Cheers,Merlyn ShelleyGrowth Lead, PacktThe Value of Thoughtful Dashboard Design in Tableau - by Ayushi BulaniIn the rush to build a new Tableau dashboard, it’s tempting to jump straight into charts and data. But taking a step back to define your dashboard’s purpose and strategy can make the difference between a report that confuses and one that doesn’t. Put simply, effective dashboards are rooted in clear objectives and an understanding of what your audience needs at a glance. (src)A common professional setting for Tableau users is the executives wanting quick insights without having to wade through noise, the analysts needing interactive exploration, and the broader audiences needing a narrative to make data relatable. A thoughtful dashboard design strategy aligns your Tableau visuals with these needs. (src) It ensures you’re not just throwing data on a page, but actually communicating the ideas. In the long run, a bit of planning on “dashboard strategy” saves time and elevates the impact of your work.Four approaches to dashboard designOne of the key insights from the upcoming book Learning Tableau 2025 is that there isn’t a one-size-fits-all approach to dashboard design. The book’s authors outline at least four common design approaches, each suited to different scenarios. Lightly adapted from Learning Tableau 2025, here are the four approaches and what they entail:🔹Guided Analysis – This approach guides the audience through the data to facilitate discovery. In practice, you lead viewers step-by-step so they can understand the data’s implications and arrive at clear actions. A guided dashboard often anticipates a specific analysis path – you’ve done the analysis and now walk the user through those findings in a logical sequence.🔹Exploratory – An exploratory dashboard is an open sandbox. It provides tools (filters, drill-downs, etc.) for the audience to explore the data on their own. The idea is that the data’s story may evolve over time, so you empower users to investigate trends and relationships themselves. This approach is common in self-service BI scenarios, where different users might have different questions.🔹Scorecard / Status Snapshot – This is all about at-a-glance information. A scorecard or status snapshot delivers a concise summary of key performance indicators (KPIs) and metrics. It’s the classic executive dashboard: think of a one-page layout with big numbers, up/down arrows, and color-coded indicators. The goal is quick problem identification and monitoring – no heavy narrative, just the vital signs of the business in one view.🔹Narrative – A narrative dashboard focuses on telling a story with the data. It guides the viewer through a beginning, middle, and end using visuals and text in a cohesive sequence. For example, you might show how a metric changed over time during a specific event (imagine illustrating the spread of a disease or the timeline of a marketing campaign). This approach adds context and commentary to data, making the insights memorable and compelling.(Extracted and adapted from Learning Tableau 2025 by Milligan et al.)Putting these approaches into practiceThese different approaches matter because of their impact. Matching your dashboard design to your audience’s needs can dramatically improve how your insights land. For instance, if your CEO just wants a daily health check of the business, a scorecard-style dashboard ensures they see all critical KPIs in seconds (and nothing more). If you’re presenting to stakeholders at a quarterly review, a narrative dashboard with a clear storyline might be more effective – it can walk them through performance drivers and outcomes in a logical flow. On the other hand, when you’re building tools for analysts or power users, an exploratory dashboard gives them the flexibility to ask their own questions about the data. And if you’ve conducted deep analysis yourself, a guided dashboard lets you package those insights into an interactive journey, so colleagues can essentially retrace your steps and findings.Keep in mind that these approaches aren’t mutually exclusive. Often, a well-crafted dashboard will blend elements of each. You might start with a snapshot overview up top (scorecard style), then provide interactive filters for deeper exploration, and perhaps include annotations or highlights to add a mini narrative. The key is to be deliberate: know when you’re trying to simply inform versus when you need to persuade or invite exploration. By aligning the design to the goal, you avoid the common pitfalls of cluttered or directionless dashboards.In today’s data-driven environment, dashboards are a staple of communication – and thoughtful design is what separates the mediocre from the truly effective. A bit of upfront strategy about how you present information pays off with dashboards that people actually use and understand. (src) Whether you’re guiding a user through a data story or letting them dive in themselves, choosing the right approach will ensure your Tableau work delivers value, not just charts.For those who want to dive deeper and see these principles in action, the book Learning Tableau 2025 is packed with practical examples and tips on building impactful dashboards. It’s a resource well worth exploring if you’re looking to sharpen your Tableau skills and design more thoughtful, effective dashboards. By approaching your next project with a clear strategy in mind, you’ll be well on your way to creating dashboards that not only look good, but drive smarter decisions in your organization.Want to design dashboards that communicate, not just display?Take the Tableau dashboard design quiz to find your weak point—and see how Learning Tableau 2025 can help you fix it. Take the quiz here!Then, pre-order your copy of Learning Tableau 2025 to learn how to apply guided analysis, exploratory tools, executive snapshots, and narrative techniques in real projects—so your dashboards deliver insight with impact.🛒 Pre-order here.⚡ What’s New in BI🔵Decoupling Semantic Model for Mirroring Customers: Semantic models are no longer auto coupled with new Mirrored artifacts. This decoupling unlocks custom semantic design, version control, and direct access to raw data, empowering teams to shape business logic independently. Phase 1 is live now; Phase 2 will decouple existing artifacts soon. This shift enhances flexibility, clarity, and scalability across analytics workflows.🔵On Adding a Start Value to a Waterfall Chart in Power BI: Power BI waterfall charts don’t natively support a custom start value, essential for tracking cumulative metrics like customer growth. This post walks through a practical workaround using a secondary date table and smart DAX measures to inject a year-end starting point. Ideal for clearer visuals when showing progression from a prior baseline into the current reporting period.🔵Build an AI assistant using Amazon Q Business with Amazon S3 clickable URLs: Create a document-aware AI assistant with Amazon Q Business that serves secure, clickable S3 links for traceability and responsible AI. This guide helps you set up the assistant, ingest enterprise files, and enable authenticated users to access referenced documents, without exposing S3 credentials. A hands-on walkthrough includes sample data, setup steps, and key IAM permissions.🔵What Is Data Literacy in 2025? It’s Not What You Think. Modern data literacy isn’t just about reading charts, it’s about cutting through noise, distraction, and AI-polished confusion to deliver clear, context-rich insights that drive decisions. This piece redefines what it means to be data-literate in 2025, and offers practical steps for building narratives that land, even with smart, overloaded, and impatient audiences.🔵Discover insights from Microsoft Exchange with the Microsoft Exchange connector for Amazon Q Business: Access Microsoft Exchange emails, calendars, and attachments through Amazon Q Business using its native connector, designed to streamline search, summarization, and task execution across enterprise communications. This guide walks through setup, indexing, and secure querying so teams can retrieve and act on Exchange data efficiently, with built-in IAM-based controls to ensure compliance and authorized access.🔵New agents and AI foundations for data teams: Google Cloud introduces a new era of data interaction: the agentic shift. This update showcases AI-native agents built into a unified data platform, empowering data teams with autonomous workflows, multimodal vector search, and real-time reasoning across operational and analytical systems. From BigQuery to Looker, intelligent agents now collaborate and act, transforming how businesses explore, understand, and act on their data.🔵Spanner's Columnar Engine Unites OLTP & OLAP: Spanner’s new columnar engine brings high-speed analytics directly into your transactional database, eliminating the latency, overhead, and complexity of ETL. With columnar storage and vectorized execution, it accelerates real-time insights on live data while preserving OLTP performance. Integrated with BigQuery via Data Boost, it simplifies architecture and enables federated queries that are faster, fresher, and more scalable than ever.🔵Standalone Copilot in Power BI will be turned on by default in September: Starting September 5, 2025, Power BI’s standalone Copilot, “chat with your data”, will be enabled by default for all tenants with Copilot already turned on. This full-screen, chat-based AI lets users explore any report, semantic model, or Fabric data agent they have access to using natural language, streamlining insight discovery across the Power BI ecosystem.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Dataproc for Apache Spark, AI Dataset Generator, Gamma Spectroscopy in PythonBecome an AI Generalist that makes $100K (in 16 hours)Still don’t use AI to automate your work & make big $$? You’re way behind in the AI race. But worry not:Join the World’s First 16-Hour LIVE AI Upskilling Sprint for professionals, founders, consultants & business owners like you. Register Now (Only 500 free seats)Date: Saturday and Sunday, 10 AM - 7 PM.Rated 4.9/10 by global learners – this will truly make you an AI Generalist that can build, solve & work on anything with AI.In just 16 hours & 5 sessions, you will:✅ Learn the basics of LLMs and how they work.✅ Master prompt engineering for precise AI outputs.✅ Build custom GPT bots and AI agents that save you 20+ hours weekly.✅ Create high-quality images and videos for content, marketing, and branding.✅ Automate tasks and turn your AI skills into a profitable career or business.All by global experts from companies like Amazon, Microsoft, SamurAI and more. And it’s ALL. FOR. FREE. 🤯 🚀$5100+ worth of AI tools across 2 days — Day 1: 3000+ Prompt Bible, Day 2: Roadmap to make $10K/month with AI, additional bonus: Your Personal AI Toolkit Builder.Register Now (Only 500 free seats)SponsoredSubscribe|Submit a tip|Advertise with usWelcome to BIPro 107 – The Tools, Trends, and Tech Redefining Business Intelligence This Week 🚀From GPU-powered NumPy and PCI-compliant analytics to multi-agent GenAI systems and synthetic data generators, this week’s roundup captures a wave of innovation reshaping dashboards, databases, and decision-making workflows.See how a Metabase engineer transformed frustration with flat datasets into an open-source generator now powering dashboards and test workflows. Discover how sales dashboards can become truly useful when built for reps, not reports. And learn how Tableau Cloud’s new PCI-DSS 4.0 compliance unlocks secure, self-serve analytics for financial teams.Take a closer look at OpenAI’s latest economic research on workplace productivity, explore a machine learning-powered gamma spectroscopy project, and find out how Google Cloud is embedding vector search and LLMs directly into Cloud SQL with nothing more than SQL.From an AI clinical copilot reducing diagnostic errors in Kenya to NVIDIA’s lightning-fast cuNumeric for GPU-accelerated NumPy, and BeyondTrust’s 30-minute QuickSight dashboard deployments, BI is becoming more live, secure, and production-ready than ever.Try the new ChatGPT agent that not only understands but acts on your requests, automating tasks like building decks and analyzing competitors. Explore Google Cloud’s curated GenAI how-to guides, and learn why Dataproc’s Lightning Engine is setting a new standard for Spark-based analytics and machine learning.Scroll down for this week’s highlights and let us know what breakthroughs or tools caught your eye.Sponsored: Your data, built your way with Twilio Segment — a customer data platform designed to cut through the chaos, unify your stack, and free you to focus on innovation over integration. Learn more.Cheers,Merlyn ShelleyGrowth Lead, Packt📊 Data Viz Trends Shaping the Future of Insights⚫🔵 The story behind our AI Dataset Generator: Frustrated by uninspiring Kaggle datasets and flawed ChatGPT outputs, a Metabase engineer built an open-source fake data generator. It uses LLMs (OpenAI, Claude, Gemini) to create realistic schemas and Faker.js to generate fast, logic-rich rows locally. With 600+ GitHub stars and Hacker News buzz, it's now a go-to tool for demos, dashboards, and testing data workflows.⚫🔵 How to build sales dashboards that sales teams actually use? Most sales dashboards get ignored because they’re built for QBRs, not quotas. A good dashboard supports reps with real-time, explorable insights, like who to call or what’s stuck. Metabase nails this with fast, self-serve, no-SQL tools and key metrics like pipeline by stage, win rate, and forecasted commissions. Build fast, listen hard, and watch usage beat requirements, every time.⚫🔵 Keep Payment and Cardholder Data Secure with PCI-DSS Compliance for Tableau Cloud: Tableau Cloud is now PCI-DSS 4.0 compliant, making it a trusted platform for securely handling cardholder data. As a Level 1 service provider, it meets top-tier security standards and empowers customers with tools like CMEK, Activity Logs, and Row-Level Security. Built on AWS and Hyperforce, it’s a secure, shared-responsibility model, perfect for financial teams seeking compliant, self-service analytics.📈 Dive into Databases: SQL Essentials⚫🔵 OpenAI’s new economic analysis: ChatGPT, now used by over 500 million people, is reshaping work, from saving teachers hours weekly to boosting public service productivity. OpenAI’s economic team has launched a deep dive into AI’s workplace impact, with new research underway. As AI scales human creativity and decision-making, the focus now shifts to ensuring the benefits are widely shared, not just concentrated.⚫🔵 Exploratory Data Analysis: Gamma Spectroscopy in Python. This project explores how machine learning can classify radioactive elements using gamma spectroscopy data. With a Radiacode detector and Python, Dmitrii Eliuseev collects spectral data, smooths and normalizes it, extracts isotope features, and trains an XGBoost model. The result: a real-time, hardware-integrated system that identifies radioactive materials, turning atomic-level radiation patterns into actionable insights for science and safety.⚫🔵 Integrate your Cloud SQL for MySQL instance with Vertex AI and vector search: Google Cloud now lets you embed and search vectors directly in Cloud SQL for MySQL using Vertex AI, no external services needed. With simple SQL, you can store embeddings, build ANN indexes, and use LLMs like Gemini for predictions and sentiment analysis. It’s a powerful way to bring semantic search and AI-driven insights straight into your application’s database layer.🔄 Real-World Transformation: How Gen BI Made Data Work⚫🔵 Pioneering an AI clinical copilot with Penda Health: OpenAI and Penda Health partnered to test a real-world LLM-powered clinical copilot across 40,000 visits in Kenya. The AI assistant, integrated into clinician workflows, reduced diagnostic errors by 16% and treatment errors by 13%. Designed with clinician input and safety in mind, AI Consult proves how well-deployed AI can meaningfully improve care quality, even in complex primary care settings.⚫🔵 NumPy API on a GPU? NVIDIA’s cuNumeric, a drop-in GPU-accelerated replacement for NumPy, is here, and it’s fast. Built on the Legate framework, it runs Python numerical code across CPUs, GPUs, and clusters with no rewrites. Benchmarks show up to 10x speedups on matrix ops. With minimal setup, data scientists can scale existing NumPy workflows effortlessly into multi-GPU territory. Python’s high-performance future is already running.⚫🔵 How BeyondTrust embedded Amazon QuickSight for identity security insights? BeyondTrust transformed its identity security reporting by embedding Amazon QuickSight into its product. With robust CI/CD pipelines, multi-tenant security, and custom UX, dashboards now deploy in under 30 minutes, slashing dev time by 89% and cutting costs by 60%. Their standout: a risk assessment dashboard built in a week. QuickSight is now central to scaling secure, insightful analytics across their platform.⚡ Quick Wins: BI Hacks for Instant Impact⚫🔵 Introducing ChatGPT agent: bridging research and action. ChatGPT now goes beyond chat, its new agent can autonomously browse, analyze, code, and deliver outputs like slides and spreadsheets using its own virtual computer. From briefing meetings to building financial models, it handles complex, multi-step tasks. With tool-switching, browser access, and human-in-the-loop controls, this upgrade makes ChatGPT a proactive collaborator for real-world workflows, outperforming both experts and prior models.⚫🔵 25 top how-to guides for Google Cloud: Google Cloud released a curated list of 25+ GenAI how-to guides for enterprises, covering model deployment, multi-agent systems, RAG pipelines, fine-tuning, and real-world app integrations. Whether you're building with Gemini, LangGraph, or Vertex AI, these recipes help you move from prototype to production faster. It's a practical, evolving toolbox for developers scaling GenAI across enterprise use cases.⚫🔵 Why use Dataproc for your Apache Spark environment? Dataproc just leveled up Spark on Google Cloud. With the Lightning Engine, you get up to 3.6x performance boosts, seamless BigQuery and GCS integration, GPU-accelerated ML, and secure, zero-scale clusters, all without infrastructure headaches. Supporting open lakehouses and fine-grained enterprise security, Dataproc is fast becoming the go-to Spark engine for AI-native, cloud-first analytics and ML workflows.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Practical strategies for shipping ML systems that work, not just demos.Subscribe|Submit a tip|Advertise with usWelcome to BIPro 106.The Machine Learning Summit opened yesterday by tackling the question a lot of teams are quietly asking:Why do so many GenAI projects stall out after the pilot phase?Stephen Klein (Curiouser.ai founder, AI ethics expert) laid it out clearly:Most failures aren’t technical. They happen because of misaligned goals, poor handoffs, and internal gridlock.The real differentiator isn’t just model performance. It’s how clearly your team can think, how quickly they can act, and how much your organization actually trusts the output.His line summed it up: “We can’t automate trust. We can’t automate imagination. That’s where the value is.”This week’s sessions are built to help you avoid those traps. Ticket holders get full access to all session replays and condensed highlights.Register for the ML SummitHere’s what’s coming up:Day 2 (today): Practical playbooks for scaling ML beyond the prototype: LLM fine-tuning, tabular and time series use cases that actually deliver value, and getting model interpretability right.Day 3: How to operationalize GenAI properly: LLMOps, causal AI, synthetic data pipelines, plus a hands-on workshop to build working AI agents.If you want frameworks and examples you can apply directly, especially for getting past stalled pilots and messy deployments, this is worth your time.Register for the ML SummitCheers,Merlyn ShelleyGrowth Lead, Packt*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

74% of firms say they’re data-driven. Only 29% actually act on it.An Exclusive Look into Next Gen BI – Live WebinarDashboards alone aren’t cutting it. The market’s moving toward something new: data apps, live collaboration, and AI that works the way teams actually work.See what's driving the rise of Next Gen BI, how Sigma earned a top debut on the Gartner Magic Quadrant, and what’s next for our roadmap.Secure Your SpotSponsoredSubscribe|Submit a tip|Advertise with usWelcome to BIPro 105 — Special EditionYes, you're seeing that right: two issues in a single week.We know that’s not our usual cadence. But this moment in BI isn’t usual either.As Generative BI accelerates and reshapes how teams interact with data, we’re testing a new format: deep dives that cut through the noise and go straight to what matters. This isn't just about keeping up with trends, it’s about keeping up with the urgency we hear from BI engineers, data leaders, and product teams across the ecosystem.We’re paying attention to what’s shifting, and this edition is us responding in real time.Because sometimes what you need isn’t more content, it’s more context.🔍 Deep Dive: Ask, Don’t Analyze, The Gen BI Era Is HereThis isn’t your average BI update. Thisis a special edition that explores one of the most critical transformations in analytics today: the shift from dashboards to dialogue.We’re moving past static reports and into natural language interactions, where insights are surfaced through simple questions, and delivered with speed, clarity, and relevance. This deep dive explores:🔍 Why traditional BI is quietly falling short🧠 How Gen BI is redefining the way insights are delivered🚀 What this means for the evolving role of BI engineersIf you're thinking about how to scale insight, reduce overhead, or future-proof your stack, this one's for you.👉 It’s not just a trend. It’s a turning point.✉️ Have tips or tools to share? Reply and contribute to our next edition.Cheers,Merlyn ShelleyGrowth Lead, PacktAsk, Don’t Analyze: How Gen BI Is Replacing Static ReportsEmpowering teams with live data and actionable stories, not static charts.“Today’s diversity and scale of BI and analytic use cases require new technology, development, and deployment options that can span from on premises to in the cloud and support data and analytic workflows at high scale and security,” said Bill Hostmann, Research Fellow, Dresner Advisory Services.This evolving landscape demands more than just traditional tools and platforms. Companies need systems that not only adapt to diverse and rapidly changing data needs, but also offer effortless scalability, security, and real-time features. Gen BI rises to meet these challenges by eliminating the barriers between users and data. It empowers decision-makers with direct, dynamic access to real-time insights, breaking down the complexity and delay often associated with older, static BI systems. With Gen BI, organizations can move faster, make smarter decisions, and stay ahead of the competition in a world where data drives success.The BI Bottleneck That No One Wants to AdmitHave you ever spent weeks building a dashboard, only to find out no one really used it? You're not alone.Thousands of BI teams across the globe suffer from the same dilemma. So, Forrester found that 74% of companies say they want to be data-driven, but only 29% actually manage to turn their analytics into action. Sounds wild, right? It’s not bad data or ugly dashboards that are the issue, it’s the whole way we’re interacting with data.People ask for dashboards, self-serve portals... but what they really need are stories. You know, something they can actually use.So, why are we still giving them static dashboards? Why are we forcing people to analyze when they just want to ask?The Stagnation of Static Dashboards1. Overengineering Without OutcomesWe’ve all been there. Weeks of development go into designing dashboards that look impeccable; interactive charts, custom filters, slicers, drill-downs. Yet when it’s finally shared, what happens? Users export it to Excel and send follow-up emails asking for help making sense of the numbers.According to a 2022 survey by NewVantage Partners, over 97% of companies say they are increasing investment in data initiatives, but only 26.5% report having a data-driven culture. The implication? We're building tools faster than people are learning to use them effectively.The real irony is that dashboards, once a symbol of democratized data access, are now gatekeepers of understanding. They often reflect more about what data is available than what insight is needed. Like a well-designed PowerPoint deck with no compelling story, static dashboards are attractive but functionally hollow.2. Cognitive Load and Interpretation GapsMost business stakeholders? They’re not data analysts. They’re decision-makers, trying to move fast.But dashboards expect them to think like statisticians: select dimensions, define filters, evaluate trends, avoid correlation fallacies.It’s like asking a pilot to diagnose engine issues mid-flight using a panel of unlabeled dials. The tools assume a level of comfort with data vocabulary that simply doesn’t exist across the business. Research from Gartner shows that data literacy remains a top barrier, with32% of business executives notfeeling confident interpreting and using data effectively.The result? Misinterpretations, overreliance on analysts, and critical delays in decision-making. We’re creating insight silos because we’re forcing everyone to self-navigate complex BI environments.3. Too Many Tools, Not Enough DecisionsIt starts small, one team swears by Power BI, another sticks with Tableau, marketing goes with Looker, and ops builds their own dashboard system. Fast forward a year, and you’ve got five BI tools, fifteen redundant dashboards, and zero alignment.Tool sprawl introduces inconsistency and confusion. Metrics are defined differently across platforms, dashboards are out of sync, and trust in data erodes. A McKinsey report found that an average of 30% of time is lost due to ineffective data governance across platforms. This leads to fragmented decision-making, inconsistent data access, and rising technical debt, all of which create inefficiencies and prevent teams from focusing on high-value tasks. As a result, organizations struggle to make timely, data-driven decisions, slowing down overall productivity.Gartner predicts that by 2025, 70% of companies will shift from big data to small, wide data, focusing on context-rich, targeted datasets that actually mean something. And guess what? That’s where Generative BI really shines.Rather than building yet another dashboard, the shift should be toward building centralized semantic layers, shared definitions, and flexible AI interfaces. Gen BI provides that bridge: fewer tools, smarter access, and consistent insight delivery.Enter Gen BI: A Paradigm ShiftIf traditional BI was about delivering data, then Generative BI (Gen BI) is about delivering understanding. It’s a big shift in how businesses interact with information. Ratherhaving to dig through dashboards, Gen BI lets you just ask what you need to know, in plain language.Let’s picturethis: You find yourself fiddling with 12 filters to track regional sales trends, or else you could just ask, “How did the Northeast do compared to the Midwest last quarter, and why?” Having to see not just a static chart or spreadsheet, the Gen BI system gives you a clear, conversational answer, with live data and visuals that actually help make sense of it.It’s a complete transformation, BI moves from being a self-service thing to a self-explaining tool. No more clicks, just conversations. No more endless exploration, just clear guidance.What Powers Gen BI?Let’s break down what’s really driving Gen BI and why it’s such a game-changer.⬛ First, we have Large Language Models like GPT-4o, Claude, or Mistral. These aren’t just fancy algorithms. This is like having a super helpful mentor or a resource personwho actually understands what you're asking. It's more like not gettinglost in complex dashboards or endless data sheets, rather you just ask your question out loud, and the model turns it into a meaningful answer. Think about how much time we spend trying to decode charts or filter through endless rows of data. LLMs take that frustration away by giving you an answer that makes sense, quickly.⬛ Then there’s the Semantic Layer. This is a huge one, especially when you're working with a lot of different teams and data sources. The issue we’ve all run into at some point is getting conflicting numbers or terms that just don’t align. Tools like dbt, Cube.dev, and LookML fix this by ensuring your data and business logic are consistent across all tools. It’s like everyone finally speaking the same language. When you ask a question, you don’t get a different answer depending on where it came from.⬛ Next up, RAG systems. These systems pull in fresh, live data from your structured sources in real time. No more working off last week's numbers or waiting for that quarterly report to be updated. If you’re asking for data now, you’re getting it now. This kind of speed is critical for real-time decision-making, and it’s one of those things that makes all the difference in a fast-paced environment.⬛ And here’s where it gets really cool: Natural Language Query (NLQ) engines. I’m sure you’ve heard of tools like Seek AI, ThoughtSpot, or Vanna.ai. These let you ask questions in plain English, no need to understand SQL or spend hours building out queries. You can just ask, “What’s my sales this month?” and it figures out the exact query needed to pull that info. That’s huge for anyone who isn’t a data scientist but still needs quick, actionable insights.In a nutshell, Gen BI makes working with data more intuitive, more aligned with your business needs, and way faster. It’s taking all those little annoyances and friction points we’ve dealt with for years and smoothing them out. We’re moving from static, report-heavy environments to something far more dynamic where you ask a question and get exactly what you need, when you need it.Who’s Leading the Charge?You’ve probably been hearing a lot about Gen BI tools lately, but who's actually making it happen in the real world? Well, the big players are already jumping in, and they’re seeing some pretty impressive results.⬛ Take Microsoft Copilot for Power BI. Ever wish you could just ask your BI tool a question in plain English and get a report back instantly? That’s exactly what Microsoft is offering now. No more sifting through endless filters and dashboards. You can just type your question in natural language, and Copilot will generate a report for you right there within Power BI. It’s almost like having a personal assistant who just gets what you need. How much time could that save you in a typical workday?⬛ Then there’s Narrative BI, a tool that turns your raw metrics into actual stories. Sounds simple, but it's a game-changer. Imagine getting a summary of your key metrics, not as dry numbers but as a story that’s easy to digest. And it doesn’t stop there, it delivers this info right to you via Slack or email. It’s like having an automatic briefing in your inbox whenever you need it. If you’re someone who’s constantly hopping between meetings, wouldn't it be great to just have those insights show up where you already work, instead of wasting time logging into a tool?⬛ And let’s not forget Airbnb’s internal AI layer. They’ve built something pretty powerful here. Product managers and marketers can ask open-ended questions about live data and get summarized insights in response. It's not about just pulling static reports from a database. It’s about asking real-time questions and getting actionable answers based on the most current data. Think about how often you’ve had to make decisions based on stale or incomplete data, Airbnb’s system is changing that, giving their teams the ability to act on insights as they come in.These companies aren’t just experimenting with Gen BI, they’re leading the way, showing us what’s possible when data becomes more accessible, faster, and smarter. It’s about freeing up time, making better decisions, and ultimately, getting things done more efficiently. Who wouldn’t want that? Read the full article on Packt’s Medium. If you’re new, make sure to follow our Medium handle and subscribe to our newsletter for more insights like this!See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

The Rise of Context Engineering, Migrating Off Kafka Connect? Meet Amazon Data FirehoseTogether with Growth School & Infinite UptimeJoin this 16 hour AI Learning Sprint to become an AI Genius (worth $895 but $0 today)The AI race is getting faster & dirtier day by day. Things we could never have imagined are happening.Thousands of people are getting laid off everydayPeople are building 1-person million-dollar companiesTech giants are fighting for AI talentMeta just poached OpenAI’s 4 top researchers …….So if you’re not learning AI today, you probably won't have a job in the next 6 months.That’s why, you need to join the 3-Day Free AI Mastermind by Outskill which comes with 16 hours of intensive training on AI frameworks, building with sessions, creating images and videos etc. that will make you an AI expert. Originally priced at $895, but the first 100 of you get in for completely FREE! Extended 4th of july SALE! 🎁📅FRI-SAT-SUN- Kick Off Call & Live Sessions🕜10AM EST to 7PM EST✅ trusted by 4M+ learnersIn the 5 sessions, you will:✅ Master prompt engineering to get the best out of AI.✅ Build custom GPT bots & AI agents for email management to save you 20+ hours weekly.✅ Create high-quality images and videos for PPTs, marketing, and branding.✅ Monetise your AI skills into a $10,000/mo business.All by global experts from companies like Amazon, Microsoft, SamurAI and more. And it’s ALL. FOR. FREE. 🤯 🚀Join now and get $5100+ in additional bonuses$5100+ worth of AI tools across 3 days — Day 1: 3000+ Prompt Bible, Day 2: Roadmap to make $10K/month with AI, Day 3: Your Personal AI Toolkit Builder.SponsoredSubscribe|Submit a tip|Advertise with usWelcome to BIPro #104 ~ rethinking enterprise intelligence for the GenAI era.This issue marks a pivotal moment in how we think about intelligence, not just artificial, but organizational. In our featured deep dive, Rahul Singh, Data Science Manager at Adobe, explores the fast-emerging discipline of Context Engineering in his groundbreaking piece:“Beyond Prompts: The Rise of Context Engineering”.As enterprises transition from assistive chatbots to autonomous AI agents, Rahul makes a compelling case: context, not prompt design, is the true foundation of effective, scalable AI. Drawing on real-world insights and examples from McKinsey, LangChain, and organizations’ own data infrastructure, this piece offers a practical roadmap for building AI systems that understand, reason, and act with enterprise-grade intelligence.Whether you're an analytics leader, a data professional, or building GenAI infrastructure, this is one article you don’t want to miss.Also in This Issue:🚀 Microsoft Fabric's June 2025 UpdatePower BI turns 10 with a celebration of features: Variable Libraries, Copilot-powered Notebooks, smarter ingestion, and real-time analytics boosts across the board.⚙️ Copy Job Upgrade: Incremental Copy Goes GANow production-ready, Copy Job’s Incremental Copy, Lakehouse Upserts, and 15+ new connectors bring faster, smarter data movement, with seamless support from on-prem to Snowflake.📉 Cloud SQL Delivers 246% ROI, Says IDCGoogle Cloud SQL shines in IDC’s latest report: faster deployments, 28% cost reductions, and over $21M in annual gains per org. Managed cloud DBs are officially a no-brainer.🧠 Meet ObjectRef: Multimodal Data Meets BigQueryGoogle introduces native unstructured data handling in BigQuery. From documents to audio and images, process, analyze, and build GenAI workflows directly in SQL or Python.🏗️ Stifel’s Event-Driven Data Mesh on AWSStifel shows how to decentralize data ownership and enable real-time updates using AWS Glue, EventBridge, and Lake Formation. A modern blueprint for financial data ecosystems.🔥 Migrating Off Kafka Connect? Meet Amazon Data FirehoseFirehose now supports custom offset timestamps, cutting complexity and cost with a fully managed, serverless path from Amazon MSK to S3.🔧 Leadership Watch: Arvind Krishnan Joins Alteryx as CTOWith deep experience at Salesforce and Bluecore, Arvind’s appointment signals Alteryx’s push into scalable, AI-powered platforms with a focus on governed innovation.BIPro 104 is your guide to the future of data platforms, context-aware AI, and modern orchestration. As GenAI systems evolve beyond simple chat prompts, the need for deeply integrated, domain-specific context is the next critical unlock. Let's build it, intelligently, and together.✉️ Have tips or tools to share? Reply and contribute to our next edition.Cheers,Merlyn ShelleyGrowth Lead, PacktUnlock 99.97% Availability with PlantOS: Production Reliability, RedefinedPlantOS Manufacturing Intelligence is powering the next era of industrial performance — delivering 99.97% equipment availability and up to 2% energy savings per unit produced. From steel to cement, manufacturers worldwide are turning fragmented data into confident decisions across every layer of production — from parameter to plant to global scale.Experience Infinite Uptime NowSponsoredBeyond Prompts: The Rise of Context EngineeringWhy context engineering is the next frontier in building smarter, more reliable AI systems.Written by Rahul Singh, Data Science Manager @Adobe. Over my seven-plus-year career in data science, working on projects ranging from customer-value measurement to product analytics and personalization, one question has remained constant through it all:Do we have the right data, and can we trust it?With the rapid rise of Generative AI, that question hasn’t disappeared; it’s become even more urgent. As AI systems evolve from proof-of-concept assistive chatbots to autonomous agents capable of reasoning and acting, their success increasingly depends not on how complex or powerful they are, but on how well they understand the context in which they operate.In recent weeks, leaders like Tobi Lütke (CEO of Shopify), Andrej Karpathy (former Director of AI at Tesla), and others have spotlighted this shift. Lütke’s tweet was widely reshared, including by Karpathy, who elaborated on it further. He emphasized that context engineering is not about simple prompting, but about carefully curating, compressing, and sequencing the right mix of task instructions, examples, data, tools, and system states to guide intelligent behavior. This emerging discipline, still poorly understood in most organizations, is quickly becoming foundational to any serious application of generative AI.This growing attention tocontext engineeringsignals a broader shift underway in the AI landscape. For much of the past year,prompt engineeringdominated the conversation, shaping new job titles and driving a surge in hiring interest. But that momentum is tapering. A Microsoft survey across 31 countries recently ranked “Prompt Engineer” near the bottom of roles companies plan to hire(Source).Job search trends reflect the change as well: according to Indeed, prompt-related job searches have dropped from144 per milliontojust 20–30(Source).But this decline doesn’t signal the death of prompt engineering by any means. Instead, it reflects a field in transition. As use cases evolve from assistive to agentic AI, ones that can plan, reason, and act autonomously, the core challenge is no longer just about phrasing a good prompt. It’s about whether the model has the right information, at the right time, to reason and take meaningful action.This is where Context Engineering comes in!Suppose prompt engineering is about writing the recipe, carefully phrased, logically structured, and goal-directed. In that case,context engineeringis about stocking the pantry, prepping the key ingredients, and ensuring the model remembers what’s already been cooked. It’s the discipline of designing systems that feed the model relevant data, documentation, code, policies, and prior knowledge, not just once, but continuously and reliably.In enterprises, where critical knowledge is often proprietary and fragmented across various platforms, including SharePoint folders, Jira tickets, Wiki pages, Slack threads, Git Repositories, emails, and dozens of internal tools, the bottleneck for driving impact with AI is rarely the prompt. It’s the missing ingredients from the pantry, the right data, delivered at the right moment, in the right format. Even the most carefully crafted prompt will fall flat if the model lacks access to the organizational context that makes the request meaningful, relevant, and actionable.And as today’s LLMs evolve intoLarge Reasoning Models(LRM), and agentic systems begin performing real, business-critical tasks, context becomes the core differentiator. Models like OpenAI’s o3 and Anthropic’s Claude Opus 4 can handle hundreds of thousands of tokens in one go. But sheer capacity is not enough to guarantee success. What matters is selectively injecting the right slices of enterprise knowledge: source code, data schemas, metrics, KPIs, compliance rules, naming conventions, internal policies, and more.This orchestration of context is not just document retrieval; it’s evolving into a new systems layer. Instead of simply fetching files, these systems now organize and deliver the right information at the right step, sequencing knowledge, tracking intermediate decisions, and managing memory across interactions. In more advanced setups, supporting models handle planning, summarization, or memory compression behind the scenes, helping the primary model stay focused and efficient. These architectural shifts are making it possible for AI systems to reason more effectively over time and across tasks.Without this context layer, even the best models stall on incomplete or siloed inputs. With it, they can reason fluidly across tasks, maintain continuity, and deliver compounding value with every interaction.Case in point:This isn’t just theory. One standout example comes from McKinsey. Their internal GenAI tool,Lilli,is context engineering in action. The tool unifies over 40 knowledge repositories and 100,000+ documents into a single searchable graph. When a consultant poses a question, it retrieves the five to seven most relevant artifacts, generates an executive summary, and even points to in-house experts for follow-up. This retrieval-plus-synthesis loop has driven ~72% firm-wide adoption and saves teams ~30% of the time they once spent hunting through SharePoint, wikis, and email threads, proof that the decisive edge isn’t just a bigger model, but a meticulously engineered stream of proprietary context (Source).What Does ContextActuallyMean in the Enterprise?By now, it’s clear that providing the right context is key to unlocking the full potential of AI and agentic systems inside organizations. But “context” isn’t just a document or a code snippet; it’s a multi-layered, fragmented, and evolving ecosystem. In real-world settings, it spans everything from database schemas to team ownership metadata, each layer representing a different slice of what an intelligent system needs to reason, act, and adapt effectively.Based on my experience working across hundreds of data sources and collaborating with cross-functional product, engineering, and data teams, I’ve found that most enterprise context and information fall into nine broad categories. These aren’t just a checklist; they form a mental model: each category captures a dimension of the environment that AI agents must understand, depending on the use case, to operate safely, accurately, and effectively within your organization.Read the full article on Packt’s Medium. If you’re new, make sure to follow our Medium handle and subscribe to our newsletter for more insights like this!📊 Signal & Insight: BI News You Need⭕ Fabric June 2025 Feature Summary: Microsoft Fabric's June update supercharges productivity: Power BI turns 10 with global celebrations and discounts. Fabric Notebooks now support variable libraries and Copilot-powered inline Python code suggestions. New features across Data Engineering, Science, and Warehouse boost modularity, scalability, and cost-efficiency. Real-Time Intelligence and Eventstream unlock deeper insights with enhanced AI, KQL, and SQL capabilities.⭕ Simplifying Data Ingestion with Copy job – Incremental Copy GA, Lakehouse Upserts, and New Connectors: Need a faster, smarter way to move data in Microsoft Fabric? The latest Copy job update delivers big wins: Incremental Copy is now generally available for efficient delta transfers, Lakehouse upserts simplify data merges, and 15+ new connectors expand your options. Plus, on-premises to Snowflake and Fabric Data Warehouse ingestion now works seamlessly, no workarounds needed. It’s flexibility and speed, built-in.⭕ How Cloud SQL boosts performance and cuts costs, per IDC: Struggling with performance, downtime, or scaling in your current database setup? IDC’s latest study shows how moving to Cloud SQL, Google Cloud’s fully managed service for MySQL, PostgreSQL, and SQL Server, drives real impact: 246% ROI, 28% lower costs, and $21.75M annual revenue gains. With AI integration, near-zero downtime, and faster deployments, Cloud SQL transforms database management into a growth engine.⭕ New ObjectRef data type brings unstructured data into BigQuery: Unifying structured and unstructured data just got easier. Google Cloud introduces ObjectRef in BigQuery, enabling seamless processing of multimodal data like images, audio, and documents alongside tabular data. With full SQL/Python support, native AI integration, and unified governance, ObjectRef empowers teams to build GenAI-powered pipelines using familiar tools, no silos, no infrastructure overhead, just smarter data workflows.⭕ How Stifel built a modern data platform using AWS Glue and an event-driven domain architecture: Looking to modernize your data architecture? Stifel shows how it's done, by building a scalable, domain-driven platform using AWS Glue, EventBridge, Lake Formation, and more. Their event-driven design enables real-time updates, decentralized data product ownership, and agile orchestration, boosting efficiency, customer experience, and ROI across business domains in a highly regulated financial environment.⭕ Overcome your Kafka Connect challenges with Amazon Data Firehose: Facing Kafka Connect complexity or cost? Amazon Data Firehose now lets you stream data from Amazon MSK to Amazon S3, fully managed, serverless, and with zero connectors to maintain. The latest update adds custom timestamp offsets, making Kafka Connect migrations seamless. Enjoy auto-scaling, simplified delivery, reduced lag, and lower TCO with event-driven stream processing.⭕ Alteryx: Appointing Arvind Krishnan as CTO for AI Growth: Alteryx has named Arvind Krishnan as its new CTO, signaling a bold push to scale its Alteryx One platform and AI Data Clearinghouse. With deep cloud and AI leadership from Salesforce and Bluecore, Arvind will drive innovation, platform stability, and cloud interoperability, strengthening Alteryx’s position in enterprise analytics and AI amid intensifying market competition.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Microsoft Fabric introduces MCP support for Real-Time Intelligence (RTI)Become an AI Generalist that makes $100K (in 16 hours)AI isn’t the future — it’s the present, quietly reshaping work, money, and opportunity. McKinsey says AI is set to add $13Trillion to the economy by 2030 — but also replace millions of jobs. Will you use it to get ahead, or get left behind? Don’t worry here’s exactly what you need: Join the World’s First 16-Hour LIVE AI Mastermind for professionals, founders, consultants & business owners like you.Rated 4.9/5 by 150,000 global learners – this will truly make you an AI Generalist that can build, solve & work on anything with AI.In just 16 hours & 5 sessions, you will:✅ Learn the basics of LLMs and how they work.✅ Master prompt engineering for precise AI outputs.✅ Build custom GPT bots and AI agents that save you 20+ hours weekly.✅ Create high-quality images and videos for content, marketing, and branding.✅ Automate tasks and turn your AI skills into a profitable career or business.All by global experts from companies like Amazon, Microsoft, SamurAI and more. And it’s ALL. FOR. FREE. Join now and get $5100+ in additional bonuses: 🔥$5,000+ worth of AI tools across 3 days — Day 1: 3000+ Prompt Bible, Day 2: $10K/month AI roadmap, Day 3: Personalized automation toolkit.Attend all 3 days to unlock the cherry on top — lifetime access to our private AI Slack community!Register Now (free only for the next 72 hours)SponsoredSubscribe|Submit a tip|Advertise with usWelcome to BIPro #103, your weekly digest of the top breakthroughs, practical insights, and real-world applications driving business intelligence forward. This edition covers everything from real-time AI integrations to SQL tuning tips, with direct links to help you dive deeper.Key Highlights:🔹 [Microsoft Fabric RTI gets MCP Support] – Bring AI to real-time dashboards with natural language access to live data and open APIs.🔹 [Detecting Fraud with Amazon Neptune + GraphStorm] – Combine graph analytics with GNNs for smarter fraud prevention.🔹 [BigQuery Enhanced Vectorization (Preview)] – Achieve 21× faster queries with SIMD and smart encoding, coming soon with Parquet support.🔹 [Inline Scalar UDFs in Microsoft Fabric Warehouse] – Reduce execution time by designing UDFs for automatic inlining.🔹 [Looker CI Preview] – Integrate LookML validations into pull requests to ship trusted models faster.🔹 [Tradeshift's Embedded Analytics with Amazon QuickSight + Q] – Empower users with natural language analytics in a unified B2B experience.🔹 [ANTLR-Powered SQL to DataFrame Converter] – Automate SQL-to-Spark/Pandas translations with grammar-driven conversion logic.🧠 Stay sharp, stay connected. BIPro brings you the trends that matter and the tools to act on them.✉️ Have tips or tools to share? Reply and contribute to our next edition.Cheers,Merlyn ShelleyGrowth Lead, PacktJoin us on July 19 for a 150-minute interactive MCP Workshop. Go beyond theory and learn how to build and ship real-world MCP solutions. Limited spots available! Reserve your seat today.Use Code EARLY35 for 35% off📊 Data Viz Trends Shaping the Future of Insights🔶 Introducing MCP Support for Real-Time Intelligence (RTI): Microsoft Fabric RTI now supports Model Context Protocol (MCP), enabling AI agents to access real-time data, run natural language queries, and integrate easily via open APIs. With schema discovery, plug-and-play setup, and future expansions like digital twins and proactive insights, MCP brings powerful, AI-driven analytics to real-time intelligence workflows.🔶 Use Graph Machine Learning to detect fraud with Amazon Neptune Analytics and GraphStorm: Amazon Neptune Analytics and GraphStorm empower fraud detection by combining scalable graph analytics with graph machine learning. Using Graph Neural Networks, the solution uncovers hidden fraud networks, predicts risk scores, and enriches transaction graphs with embeddings. This pipeline, deployable via SageMaker, enables real-time, dynamic analysis of evolving fraud behaviors.🔶 Understanding BigQuery enhanced vectorization: Google BigQuery’s enhanced vectorization, now in preview, redefines query performance by leveraging SIMD processing, advanced data encodings, and parallel execution. Tight integration with Capacitor enables faster joins, aggregations, and filter pushdowns. Real-world benchmarks show up to 21× speedups, delivering major gains in efficiency. Full rollout and Parquet support coming soon!📈 Dive into Databases: SQL Essentials🔶 How to make your SQL scalar user-defined function (UDF) inlineable in Microsoft Fabric Warehouse? Microsoft Fabric now supports inline scalar UDFs in Warehouse preview, improving performance by reducing execution overhead. This post highlights common inlining blockers like non-deterministic functions, complex query usage, and multiple return statements, with guidance on how to refactor them. Broaden SQL compatibility and efficiency through inline-friendly UDF design practices.🔶 Introducing Continuous Integration for Looker: Looker now offers Continuous Integration (CI) in preview, giving developers tools to improve speed, accuracy, and confidence in deploying LookML code. CI validates LookML, detects SQL and content issues early, and integrates with pull requests or schedules. This helps teams catch errors before production, ensuring reliable and consistent data experiences.🔶 Recovering data in SQL Server without full backup: When full backups fail, SQL Server's transaction logs can help recover deleted data. This guide walks through using fn_dump_dblog to extract deleted rows from .trn files, even without a full backup. It’s a complex but powerful recovery method that reveals deep insights into SQL Server internals and storage behavior.🔄 Real-World Transformation: How Gen BI Made Data Work🔶 Fabric Eventhouse now supports Eventstream Derived Streams in Direct Ingestion mode (Preview): Microsoft Fabric now supports direct ingestion of derived streams into Eventhouse, enabling seamless, no-code real-time data routing. Users can configure this from the Eventstream canvas, Embedded Real-Time Hub, or Eventhouse Get Data Wizard. Derived streams, created through transformations, can now be ingested directly into KQL tables. This feature is currently in Preview.🔶 The Hidden Cost of MAXDOP: CPU Spikes, Query Slowdowns, and What We Learned. Hardcoding MAXDOP 4 in queries boosted individual performance but caused CPU contention and system-wide slowdowns. Testing revealed that higher MAXDOP reduces concurrency on limited-core systems. The real fix came from proper indexing and statistics updates, cutting query time from 20 to 1 second. Always tune first before applying parallelism.🔶 From code to community: The collective effort behind SQL Server 2025. SQL Server 2025 is now in public preview, bringing AI integration, vector search, zero-ETL analytics via Fabric mirroring, and powerful developer features like JSON and regex support. Designed for cloud agility and built through deep community collaboration, it’s the most transformative SQL Server release in over a decade.⚡ Quick Wins: BI Hacks for Instant Impact🔶 Transforming B2B intelligence: Tradeshift’s journey with Amazon QuickSight and Amazon Q.Tradeshift transformed its B2B analytics by embedding Amazon QuickSight and Amazon Q into its platform, enabling scalable, AI-powered insights for buyers and sellers. This shift replaced a limited in-house tool, reduced engineering overhead, and empowered users to explore data through natural language, redefining how intelligence is delivered across their global e-invoicing network.🔶 Building an SQL to DataFrame Converter With ANTLR: ANTLR enables seamless SQL-to-DataFrame translation by parsing SQL syntax into structured trees, which can be programmatically converted into Pandas or Spark code. This approach tackles key challenges like syntax differences, execution models, and feature gaps. With proper grammar, visitors, and dialect handling, developers can automate and scale query conversion across frameworks.🔶 PostgreSQL: pgSCV 0.14.1 released! This PostgreSQL monitoring agent adds support for pg_stat_ssl, pg_stat_subscription, and new pg_stat_io metrics. It also updates Grafana dashboards, Go, and Alpine Linux. Designed for Prometheus compatibility, pgSCV continues to evolve as a unified exporter for PostgreSQL environments. Full details on GitHub.🔶 MySQL Orchestrator Failover Behavior During Replication Lag: Managing MySQL replication failovers becomes more resilient with orchestrator settings like FailMasterPromotionIfSQLThreadNotUpToDate, DelayMasterPromotionIfSQLThreadNotUpToDate, and FailMasterPromotionOnLagMinutes. These options offer fine-grained control over failover behavior, ensuring consistency by preventing or delaying master promotions when replicas are lagging, ultimately balancing availability and data integrity during outages or transitions.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Local Debugging for Fabric User Data Functions, BigQuery Workload Management UpgradesYour Exclusive Invite for the World’s first 2-day AI Challenge (usually $895, but $0 today)51% of companies have started using AITech giants have cut over 53,000 jobs in 2025 itselfAnd 40% of professionals fear that AI will take away their job.But here’s the real picture - companies aren't simply eliminating roles, they're hiring people who are AI-skilled, understand AI, can use AI & even build with AI. Join the online 2-Day LIVE AI Mastermind by Outskill - a hands-on bootcamp designed to make you an AI-powered professional in just 16 hours. Usually $895, but for the next 48 hours you can get in for completely FREE!In just 16 hours & 5 sessions, you will:Learn the basics of LLMs and how they workMaster prompt engineering for precise AI outputsBuild custom GPT bots and AI agents that save you 20+ hours weeklyCreate high-quality images and videos for content, marketing, and brandingAutomate tasks and turn your AI skills into a profitable career or business📅Kick off Call & Session 1- Friday (10am EST- 1pm EST)🧠Sessions 2-5:🕜Saturday 11 AM to 7 PM EST; Sunday 11AM EST to 7PM ESTAll by global experts from companies like Amazon, Microsoft, SamurAI and more. And it’s ALL. FOR. FREE. You will also unlock $3,000+ in AI bonuses: Slack community access, Your Personalised AI tool kit, and Extensive Prompt Library with 3000+ ready-to-use prompts - all free when you attend!Join in now, we have limited free seats!SponsoredSubscribe|Submit a tip|Advertise with usWelcome to BIPro #102 – Smarter, Sharper, AI-First BIFrom zero-ETL breakthroughs to smarter debugging in Fabric, this issue is packed with forward-looking updates that make your BI practice leaner and more intelligent. Whether you're exploring transactional BigQuery features, debugging user data functions locally, or scaling agentic AI, these updates give you the tools to stay ahead in an increasingly AI-driven analytics landscape.Key Highlights:Free Chapter: Learning Tableau 2025Explore Tableau’s latest AI-powered features with a hands-on chapter from the bestselling series by Joshua Milligan.BigQuery Workload Management UpgradesBoost efficiency with smarter slot allocation, autoscaling improvements, and predictable performance.Zero-ETL from DynamoDB to SageMaker LakehouseEnable real-time analytics without pipelines using no-code CDC-based integration.Local Debugging for Fabric User Data FunctionsTest and troubleshoot Python-based SQL code safely in VS Code—no production risks.Best Practices for Purview DLP in FabricSecure sensitive data in OneLake with Microsoft’s enhanced DLP policies.Azure Data Factory vs Synapse vs Fabric PipelinesGet clarity on core differences in tools, billing, and connection management.SponsoredCloudVRM slashes vendor review and audit time by connecting directly to cloud environments, no spreadsheets, no forms, just real-time compliance, 24/7. Watch the demo.💡 Whether you’re scaling AI in your stack or still fighting dashboard fatigue, this week’s BIPro is packed with insights to move your strategy forward.Helping you close the gap between data and action.Cheers,Merlyn ShelleyGrowth Lead, PacktGet Chapter 1 of Learning Tableau 2025 – Free!Explore Tableau’s newest AI-powered capabilities with a free PDF of Chapter 1 from the latest edition of the bestselling series, Learning Tableau 2025.Written by Tableau Visionary Joshua Milligan, this hands-on guide helps you build smarter dashboards, master data prep, and apply AI-driven insights.Sign up to download your free chapter!Grab Your Free Chapter Now!📊 Data Viz Trends Shaping the Future of Insights🔷 Understanding updates to BigQuery workload management: Boost BigQuery Efficiency. Google’s June 2025 update enhances BigQuery workload management with fairer slot allocation, reservation predictability, autoscaler granularity, runtime reservation targeting, and detailed cost labeling. These updates improve performance, cost control, and scalability, empowering BI teams to manage compute resources more intelligently for consistent, optimized data analytics at scale.🔷 BigQuery features for transactional data management: BigQuery Evolves Beyond Analytics. Google’s latest update brings transactional power to BigQuery. Highlights include fine-grained DML for efficient data edits, CHANGES TVF for row-level change tracking, and real-time DML on streaming data. These features streamline dynamic data management, cutting costs, reducing latency, and simplifying architectures for responsive, analytics-integrated applications.🔷 Refresh SQL analytics endpoint Metadata REST API (Preview): Refresh SQL Endpoint Metadata On-Demand: Microsoft Fabric now enables API-based table metadata refresh for SQL analytics endpoints. Gain synchronous/asynchronous control, real-time sync status, error diagnostics, and timeout customization. This streamlines data validation, enhances workflow precision, and ensures up-to-date metadata for accurate, responsive analytics across your Fabric workspace.📈 Dive into Databases: SQL Essentials🔷 How to Perform a Disaster Recovery Switchover with Patroni for PostgreSQL? Ensure PostgreSQL Resilience with Patroni DR Switchover: Learn how to perform a disaster recovery switchover using Patroni for PostgreSQL. This guide covers DC-DR cluster setup, real-time failover management, and safe primary-to-standby transitions. Streamline HA/DR operations, minimize downtime, and safeguard data continuity across distributed environments using Patroni and AnyDBver.🔷 AI Agents: The Protocol Revolution Driving Next-Gen Enterprise Intelligence. Agentic AI Standards Take Shape: New protocols, MCP, ACP, and A2A, are accelerating enterprise adoption of agentic AI. These standards simplify agent collaboration, cross-platform integration, and real-time context building. With growing vendor support, including Couchbase and Google, developers can now build scalable, intelligent, multi-agent systems more easily, ushering in AI’s next enterprise phase.🔷 Techniques to query Azure SQL’s new JSON Datatype: Azure SQL Embraces Native JSON: Azure SQL’s new JSON datatype and functions (including JSON_ARRAYAGG, JSON_OBJECTAGG) simplify structured JSON handling. Benefits include native constraints for data validation, powerful aggregation, and dynamic pivoting. These enhancements streamline semi-structured data analytics, boost SQL Server 2025 readiness, and align with ANSI SQL:2016 standards.🔄 Real-World Transformation: How Gen BI Made Data Work🔷 Simplify real-time analytics with zero-ETL from Amazon DynamoDB to Amazon SageMaker Lakehouse: Zero-ETL for Real-Time Insights: AWS introduces zero-ETL integration from DynamoDB to SageMaker Lakehouse. This no-code setup enables real-time analytics and ML on NoSQL data, without ETL pipelines. With CDC-based syncing, Apache Iceberg support, and SageMaker Unified Studio access, enterprises can drive faster, scalable decision-making from operational data.🔷 Pub/Sub single message transforms: Streamline Real-Time Data with Pub/Sub SMTs: Google Cloud introduces Pub/Sub Single Message Transforms (SMTs), starting with JavaScript UDFs. Perform lightweight in-stream transformations, like redaction, format conversion, and filtering, directly in Pub/Sub. This reduces latency, simplifies architecture, and enhances flexibility for BI teams needing real-time insights without extra processing infrastructure.🔷 How to debug user data functions locally in VS Code? Accelerate Fabric Function Development with Local Debugging: Microsoft Fabric now supports local debugging for user data functions in VS Code. Developers can write, test, and troubleshoot Python-based SQL operations safely without impacting live environments. Breakpoints, logs, and Fabric extensions enhance function validation, boosting efficiency in building robust, data-driven BI workflows.🔷 Secure Your Data from Day One: Best Practices for Success with Purview Data Loss Prevention (DLP) Policies in Microsoft Fabric. Secure Fabric Data with Purview DLP Best Practices: Microsoft Fabric now offers enhanced Data Loss Prevention (DLP) via Purview. From classifying sensitive data to refining policy scopes and empowering teams, these strategies help protect OneLake data, ensure compliance, and foster responsible data use, establishing a secure foundation for scalable analytics.⚡ Quick Wins: BI Hacks for Instant Impact🔷 SQL Automate - A DBA's Time-Saving Toolkit: Optimize SQL Server Deployments with SQL Automate: SQL Automate is a PowerShell-based toolkit that simplifies and speeds up SQL Server installation and post-configuration tasks. From database setup to TempDB tuning and firewall rules, DBAs can save time, reduce manual errors, and focus on strategic work with this efficient, modular automation tool.🔷 How to Sort Combo Box Values in a Power Apps Canvas App? Enhance Power Apps UX with Sorted Combo Boxes: Sorting Combo Box and Drop Down values in Power Apps boosts usability. This guide shows how to alphabetically or numerically sort SharePoint list-driven fields using formulas like Sort and SortByColumns, ensuring cleaner, more intuitive interfaces in forms and data-driven canvas apps.🔷 Top 5 Things You Should Know About Azure Data Factory: Master Azure Data Factory Fundamentals: With three ADF versions, Azure, Synapse, and Fabric, understanding their differences is essential. This guide covers key distinctions, connection management quirks, and pricing basics. Whether you're onboarding teammates or migrating workloads, these insights help you navigate ETL/ELT orchestration and cost-efficient pipeline development across Microsoft's evolving data platforms.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

FREE GUIDE: Airflow 3 Tips & Code SnippetsFREE GUIDE: Airflow 3 Tips & Code SnippetsThinking about upgrading to Apache Airflow® 3? You’ll get powerful new features like a modernized UI, event-based scheduling, and streamlined backfills. Quick Notes: Airflow 3 Tips & Code Snippets is a concise, code-filled guide to help you start developing DAGs in Airflow 3 today.You’ll learn:How to run Airflow 3 locally (with dark mode) and navigate the new UIHow to manage DAG versioning and write DAGs with the new @asset-oriented approachThe key architectural changes from Airflow 2 to 3GET YOUR FREE GUIDESponsoredSubscribe|Submit a tip|Advertise with UsWelcome to BIPro #111: Expert Insight EditionWe’re excited to introduce Sagar Lad, Lead Data Solution Architect at a leading Dutch bank, as our newest Expert Insight contributor. Each week, Sagar will share battle-tested lessons, practical tips, and implementation strategies for building resilient data products in the Gen AI and Agentic AI era.He kicks off this week with a deep dive into Data Products: Turning Data into Tangible Value, showing how to move from concept to measurable business impact. This marks the start of a series designed to help you apply hard-won expertise directly into your data practice.Alongside Sagar’s expert guidance, here are this week’s top stories in data & BI:🔗 Zero-Latency Data Analytics for PostgreSQL Apps – AWS introduces zero-ETL integration from RDS PostgreSQL to Redshift, enabling real-time analytics without fragile pipelines.🔗Amazon Q Developer CLI – From theory to practice: AI-powered project advisor that turns AWS certification knowledge into hands-on portfolio projects.🔗Map Visualization in BigQuery Studio – GA launch brings geospatial queries to life, letting analysts instantly visualize and interact with Earth Engine datasets.🔗Amazon Timestream for InfluxDB – Expanded global rollout with compute/storage scaling, Multi-AZ replicas, and InfluxDB 3 migration path for time series workloads.🔗Power BI Semantic Model Refresh Templates (Preview) – Orchestrate refreshes with Fabric Data Pipelines, supporting event-driven triggers, incremental refreshes, sequencing, and alerts.As always, our mission is to connect you with first-hand expert insights and timely news updates, so you can stay ahead of change while sharpening your craft.Cheers,Merlyn ShelleyGrowth Lead, PacktData Products: Turning Data into Tangible Value - By Sagar LadIn today’s digital economy, data has become one of the most valuable assets for organizations. Every transaction, interaction, and process generates data that — when properly harnessed — can unlock powerful insights, drive innovation, and create competitive advantages. However, simply collecting and storing vast amounts of data is not enough. To truly realize its value, organizations must transform data into usable, scalable, and outcome-driven solutions. This is where the concept of adata productcomes into play.A data product is not just raw data, but rather a packaged, consumable, and value-generating asset built on top of data. Just as traditional products solve customer needs, data products solve business challenges by delivering insights, predictions, or automated decisions in a way that is accessible and reliable for end users.What is a Data Product?At its core, adata productis a solution designed around data to serve a specific purpose or generate business value. It could take many forms — such as a dashboard, an API serving machine learning predictions, a recommendation engine, or even a dataset curated for a particular domain.For example:→ Netflix’s recommendation systemis a data product built to enhance user engagement.Characteristics of a data product include:1. Purpose-driven— It is built to achieve a clear outcome (e.g., increase sales, reduce costs, improve customer satisfaction).2. Reusable— A well-designed data product can serve multiple teams or applications.3. Consumable— It is packaged in a way that non-technical users or systems can leverage it seamlessly.4. Scalable— It is designed to evolve with changing business needs and data volumes.Data Product: Bridge between Producer & ConsumerData Products vs. Data AssetsIt is important to differentiate betweendata assetsanddata products.Adata assetcould be a data lake, warehouse, or dataset that stores raw or processed data. While valuable, assets by themselves may not generate outcomes unless someone analyzes them.Adata product, on the other hand, transforms these assets into actionable, consumable outputs that stakeholders can directly use to make decisions or power business processes.In other words, data assets are ingredients, while data products are the finished dishes that customers can consume.Why Do Organizations Need Data Products?Organizations often struggle with extracting value from their data investments. Billions of dollars are spent globally on data platforms, yet many businesses face the“last mile problem”— where insights fail to reach decision-makers in a meaningful way. Data products help bridge this gap by operationalizing data and embedding it into workflows.Key benefits of data products include:1. Faster Decision-MakingWith well-packaged insights, business users don’t need to spend hours querying databases or waiting for reports. A data product like a sales forecasting model can instantly provide actionable intelligence.2. Democratization of DataData products abstract technical complexity, enabling business users, analysts, and applications to easily consume data-driven insights.3. Standardization and ReusabilityInstead of rebuilding analytics pipelines repeatedly, a single data product can serve multiple business units. For example, a customer segmentation data product could be reused by marketing, sales, and product teams.4. Scalability and AutomationData products, once designed, can be scaled to handle growing data volumes and embedded into automated workflows.5. Value RealizationUltimately, data products help organizations move beyond storing data tomonetizing and operationalizing it— whether through cost savings, revenue generation, or improved customer experiences.Key Principles for Designing Data ProductsDesigning a successful data product requires more than technical skills — it requires product thinking. Some guiding principles include:1.Start with Business ValueA data product must solve a real business problem. Before building, clearly define the outcome it should drive.2. User-Centric DesignThe product should be intuitive for its target users, whether that’s executives, developers, or customers.3. Trust & TransparencyUsers must trust the data product. This requires data quality checks, explainability in AI models, and governance measures.4. Scalability & ReusabilityBuild products that can adapt to future needs, serve multiple stakeholders, and scale across datasets and domains.5. OperationalizationA data product should integrate seamlessly into business workflows and systems, rather than existing as a standalone artifact.6. Monitoring & ImprovementData products must be continuously monitored for performance, accuracy, and relevance, with feedback loops for improvements.Challenges in Building Data ProductsWhile data products are powerful, organizations face challenges in creating and scaling them:1. Data Quality Issues: Poor data leads to unreliable products.2. Cultural Resistance: Teams may hesitate to trust automated insights.3. Lack of Product Mindset: Many companies treat data as IT projects, not products.4. Scalability Hurdles: A data product may work for a pilot but struggle in enterprise-wide deployments.5. Governance & Compliance: Ensuring data products adhere to regulatory and ethical standards is critical.Overcoming these requires strongdata governance, clear ownership, cross-functional collaboration, and a product-centric approach.Read the full article on our Packt Medium page, and don’t forget to follow us for more expert insights like this.💡 Smarter Insights This Week🔳 Power BI August 2025 Feature Summary: What if Power BI could think smarter for you? The August 2025 Feature Summary delivers Copilot in SharePoint Online, automated measure descriptions, and filtered report summaries. Plus, Pro workspace org apps, advanced modeling with Databricks Direct Lake, new connectors, and visual upgrades, all boosting speed, scale, and smarter insights.🔳 Benchmarking AWS S3 Performance With Python Scripts: Ever wondered if your AWS S3 storage is really as fast or cheap, as promised? This guide shows how Python benchmarking uncovers latency and throughput trade-offs across storage classes. Learn setup, key metrics, practical scripts, and best practices to balance cost and performance while avoiding hidden bottlenecks.🔳 Zero-Latency Data Analytics for Modern PostgreSQL Apps: Tired of building fragile data pipelines? AWS now offers zero-ETL integration from RDS PostgreSQL to Redshift, announced July 23, 2025. Stream transactional data in seconds, apply filters per integration, and even automate setup with CloudFormation, unlocking real-time analytics and ML without the overhead of ETL.🔳 Using Google’s LangExtract and Gemma for Structured Data Extraction: Struggling to make sense of dense documents? Google’s LangExtract framework with Gemma 3 LLM makes structured data extraction from long unstructured text faster, smarter, and traceable. With chunking, parallel processing, and iterative passes, it surfaces key facts, like insurance policy exclusions, turning legalese into structured, plain-English insights you can actually use.🔳 From theory to practice using Amazon Q Developer CLI to generate tailored AWS projects: Too much AWS theory, not enough practice? That’s where most learners stall. Amazon’s Q Developer CLI transforms study into action by suggesting skill-matched projects and guiding implementation with CLI commands. From S3 websites to CloudFront and IaC, it makes building practical, portfolio-ready AWS projects achievable for everyone.🔳 Zero-ETL: How AWS is tackling data integration challenges? In this blog post, we explore how AWS is simplifying data integration with zero-ETL, replacing fragile pipelines with real-time replication. From Aurora, RDS, and DynamoDB to Redshift and SageMaker, zero-ETL reduces costs, handles schema changes automatically, and delivers near-instant analytics, transforming integration from an engineering burden into a strategic advantage.🔳 Amazon Timestream for InfluxDB: Expanding managed open source time series databases for data-driven insights and real-time decision making? Time series data is exploding, powering everything from IoT to gaming. AWS Timestream for InfluxDB now spans 19 Regions, offering compute/storage scaling, Multi-AZ replicas, and 24xlarge instances. Backed by a strategic partnership with InfluxData, customers gain real-time analytics, high cardinality support, and a migration path to InfluxDB 3.🔳 Firestore with MongoDB compatibility is now GA: At Google Cloud Next ’25, Firestore previewed MongoDB compatibility, and it’s now generally available. Developers can reuse MongoDB code, drivers, and tools with Firestore’s serverless database, gaining multi-region replication, 99.999% availability, PITR recovery, triggers, and enterprise-grade security. With expanded API support and Firebase access, Firestore delivers scalable, cost-efficient document storage.🔳 Earth Engine raster analytics and visualization in BigQuery geospatial: Geospatial data holds untapped business value. Now generally available, Earth Engine in BigQuery lets analysts join satellite imagery with structured data for richer insights, from disaster risk to supply chain planning. Paired with new map visualization in BigQuery Studio, it transforms complex geospatial queries into intuitive, interactive decision-making tools.🔳 Semantic Model Refresh Templates in Power BI (Preview). Keeping data models fresh can be complex. With Semantic Model Refresh Templates in Power BI (preview), you can orchestrate refreshes using Fabric Data Pipelines, supporting event-driven triggers, incremental refresh, sequencing multiple models, and automated alerts. The guided setup makes advanced refresh workflows easier, faster, and more reliable for analysts.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

The Missing Pieces in BI Automation and How Agentic AI Completes Them👋 Hello ,Welcome to BIPro Issue 121.This week’s edition brings focused insights for BI leaders and data teams navigating the shift to intelligent automation. In our latest Packt Talks conversation, Abhishek connects with Pablo Moreno, AI Product Manager at Board, to unpack one of the most persistent challenges in BI today: why analytical and operational workflows keep breaking, and how agentic AI is becoming the scalable and adaptable layer modern enterprises need.Inside this issue, we walk through the episode with a BI practitioner lens. You’ll find real examples from domains like pharma and workforce analytics, a look at how frameworks like Crew AI are reshaping orchestration, emerging AI-driven roles across data teams, and what the move from rigid, deterministic automation to self-managing agents means for anyone building or maintaining BI systems. If you work in reporting, data engineering, analytics, or BI strategy, you’ll find practical takeaways you can put to work right away.💡Workshop Spotlight:Algorithmic Trading with Python (Cohort 2)Only 10 seats left. Join Jason Strimpel on December 12 for a hands-on workshop where you’ll design, test, and deploy algorithmic trading strategies in Python with pandas, VectorBT, and the Interactive Brokers API. Whether you’re leveling up as a quant or sharpening your finance toolkit, this live session covers the full workflow from finding edges to executing real strategies. Choose a Full Pass or VIP Pass, use FINAL20 for 20 percent off, and secure your spot before it’s gone.Book Now and Save 20% – Only 10 Seats LeftHere’s Jason’s latest AMA. It gives you a quick preview of the insights and style you can expect in the workshop.This Week’s Sponsor Spotlight✔️ The First AI Employee for Your Mobile App Always On, Always Improving Your ProductFload watches your app around the clock, flags issues before they become revenue losses, uncovers new growth opportunities, and forecasts what’s coming next, all without manual effort.It’s proactive intelligence built directly into your product lifecycle. Try Fload free, connect your app in 60 seconds.Try Fload✔️ The GenAI Experience Users Want Secure, Controlled Access to Your SaaS ProductAI agents are already trying to interact with your product. With AgentLink, you stay in control.Give your users AI-powered workflows while ensuring every interaction is secure, governed, and policy-driven. Try AgentLink.Try AgentlinkCheers,Merlyn ShelleyGrowth Lead, PacktSubscribe|Submit a tip|Advertise with UsWhen Automation Breaks: How Agentic AI Solves Hidden Problems in Packt Talks with Pablo MorenoEvery company believes their automation is working, until it suddenly breaks. A small formatting change, a new policy variation, or a shift in user behavior can turn even the most expensive automated workflow into a bottleneck. This is the silent problem most businesses face: systems built to save time often create more manual work in the long run.In the latest episode ofPackt Talks, our GLAbhishek Kaushikdives into this challenge withPablo Moreno, AI Product Manager atBoard, a leader in financial planning and performance management software. Together, they unpack why traditional automation struggles to keep up with modern complexity and howagentic AIis rewriting the blueprint for scalable, resilient operations. The conversation sets the foundation for a deeper exploration of real-world cases, emerging frameworks, and the future of human-AI collaboration, insights that shape the rest of this article.The Automation Myth: Why Your Workflows Still BreakTraditional automation has helped companies save time for decades, but its biggest weakness remains unchanged: it can’t adapt. When business conditions shift, when documents look different, or when people ask slightly different questions, automated systems often fail. This mismatch between rigid workflows and fast-changing environments has widened over the last decade. Agentic AI steps in as a new paradigm, one where automation doesn’t just follow rules but makes decisions within defined boundaries.TakeawaysAutomation saves time but fails in dynamic, unpredictable scenarios.The pace of change in the last 5–10 years has outpaced deterministic workflows.Agentic AI thrives where traditional automation breaks, handling complexity without endless reprogramming.77% of enterprises are exploring agentic AI, with 30% already seeing major reductions in manual work.Problem in Focus: When Rule-Based Automation Turns into Technical DebtMany organizations rely on hyperautomation, systems built on stacks of rules, templates, and RPA bots. But every new rule becomes a maintenance cost. A single formatting change can break an automated pipeline. In the pharma industry example Pablo Moreno shares, scanning marketing documents for legal disclaimers turned into an expensive cycle of bot retraining.TakeawaysRule-heavy automation becomes brittle and expensive at scale.Minor variations (orientation, typos, layout changes) often break RPA workflows.Over-reliance on deterministic rules leads to high maintenance debt.The real cost of automation isn’t deployment — it’s constant upkeep.How Agentic AI Fixes the RPA ProblemAgentic AI doesn’t rely on rigid patterns. It understands context, recognizes variations, and adapts to change. In the pharma compliance case, agents could detect legal phrases regardless of spelling errors, text rotation, or synonyms. Instead of adding rules, the system learned patterns. This shift replaced hours of manual review and repetitive bot tuning.TakeawaysAgentic AI extracts meaning, not formatting, reducing the need for rules.Agents adapt to document variance with little intervention.Maintenance drops because the system generalizes from examples.Compliance workflows become faster, cheaper, and more reliable.Why Crew AI Works: Solving the Complexity ProblemChoosing the right framework matters. Pablo highlights Crew AI as his preferred approach because it enables multi-agent processes while retaining simplicity. Unlike low-code/no-code tools that struggle once workflows become nonlinear or multi-context, Crew AI supports scalable architectures, evolving use cases, and quick iteration cycles.TakeawaysCrew AI enables multi-agent workflows without heavy engineering overhead.Low-code/no-code tools are fast but tend to break once complexity grows.Crew AI supports new agents, tools, integrations, and iteration loops.Scalability and adaptability should guide platform decisions, not speed alone.The HR Chatbot Challenge: When Reality Is MessyA multinational pharma company attempted to build an HR chatbot for 100,000 employees across 40 countries. The complexity was enormous: policies varied by 20–30 percent, acronyms differed across teams, and employee questions were full of local nuance. The breakthrough came from building a robust knowledge base, including an acronym dictionary, and extensively training the system to recognize contextual differences.TakeawaysReal-world enterprise AI must handle messy, inconsistent data.Local nuances in HR policies can’t be hard-coded into rules.Knowledge bases and acronym dictionaries significantly improve AI accuracy.Good framing and good inputs are critical for agentic AI performance.Continue reading the full article on the Packt Medium handle.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0}#converted-body .list_block ol,#converted-body .list_block ul,.body [class~=x_list_block] ol,.body [class~=x_list_block] ul,u+.body .list_block ol,u+.body .list_block ul{padding-left:20px} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Karun explains how to build stronger intuition, clearer portfolios, and higher value projects.Subscribe|Submit a tip|Advertise with Us🧩Welcome to BIPro 120. Our Expert Insight this week features Walmart Senior Data Scientist Karun, who breaks down how AI, automation, and GenAI are rewriting the rules of business intelligence & data science. Whether you are job hunting or leveling up your skills, his advice cuts straight to what matters. We are also bringing something new to our readers, straight from experts with first-hand industry experience. Packt has launched live AMA sessions with industry leaders on the Packt DataML YouTube channel, hosted by our Growth Lead Abhishek Kaushik. These sessions give you direct access to practical guidance, career clarity, and real world insights from voices shaping the future of Python, AI, and data science.In this edition, we have included Karun’s take on evolving data science roles, portfolio strategies that stand out, and the skills hiring managers value most in 2025 and 2026. You will find the extended highlights below.Knowledge Partner Spotlight: OutskillAt Packt, we’ve partnered withOutskillto help readers gain practical exposure to AI tools through free workshops, complementing the deeper, hands-on, expert-led experiences offered throughPackt Virtual Conferences.Still doing manual tasks repetitively? Learn AI to automate 50% of your work.Outskill is hosting a 2-Day Live AI Mastermind, an intensive training on AI tools, automations, and agent building designed to make you indispensable. As part of their Holiday Season Giveaway, the first 100 participants can join for free, even though the program normally costs $395. The live sessions run Saturday and Sunday from 10 AM to 7 PM EST, and attendees unlock over $5000 worth of AI resources, including a Prompt Bible, a roadmap to monetizing with AI, and a personalized AI toolkit builder.Register here for $0 (first 100 people only)💡 Workshop Spotlight: Algorithmic Trading with Python (Cohort 2)Join Jason Strimpel for a hands-on workshop on December 12, where you’ll design, test, and deploy algorithmic trading strategies in Python using pandas, VectorBT, and the Interactive Brokers API. Whether you’re a quant-in-training or a finance pro, this live session takes you through the entire workflow, from identifying trading edges to executing real-world strategies. Pick a Full Pass or VIP Pass, use FINAL20 for 20% off, and reserve your seat.Last Chance: Register Now and Get 20% OffCheers,Merlyn ShelleyGrowth Lead, PacktThe Real Story Behind Today’s Data Science Job Market and How to Stand Out with Karun ThankachanIf you feel like the data science world is shifting under your feet, you are not alone. In the latest episode of Pack Talks, Abhishek Kaushik sat down with Karun, a senior data scientist at Walmart, to unpack what it really takes to break into and thrive in this field today. The conversation was honest, pragmatic, and sprinkled with hard-won wisdom from someone who has navigated one of the toughest job markets in recent history.This article brings you the essential takeaways: how the role of a data scientist is evolving, what skills truly matter now, how to build a competitive portfolio, and the strategies that actually work for landing interviews and surviving them.Let’s dive in.From Pandemic Turbulence to Walmart: Karun’s Nonlinear JourneyKarun’s story begins like many hopeful data scientists. He moved to the US for his master’s at Carnegie Mellon, rode the initial excitement of landing an internship, then watched that offer disappear during the COVID downturn. It was a moment when thousands of fresh graduates questioned whether the field still had space for them.Instead of giving up, he leaned into referrals, rebuilt his pipeline, and secured a data science internship at Amazon. That later became a full-time role, and today he works on high-impact data science initiatives inside Walmart.His journey is a reminder that resilience still matters. The field is changing, but it is far from closing its doors.Is Data Science Still a Good Career? The Market Has Evolved, Not ShrunkWhen asked the million-dollar question, Karun was clear. The career is still viable, but the expectations have shifted.Here is how the field has changed since 2017.1. Model building has become easierThe rise of tools like XGBoost and transformers reduced the complexity of many modeling tasks. You no longer need niche mathematical expertise to build a model that works.2. Business impact has become the differentiatorHiring managers want people who can translate problems into solutions that move metrics, reduce costs, or improve customer experience.3. Entry-level roles now demand mid-level thinkingThanks to automation and AI assistants, junior roles expect stronger system design, problem framing, and measurable impact.4. Career growth now mirrors software engineeringInstead of staying at the “modeling” layer, data scientists are expected to grow into architecture, experimentation strategy, and problem definition.The bottom line: AI did not remove data science jobs. It elevated the bar for what a data scientist does.The Four Data Science Roles You Need to Understand in 2025Karun breaks down today’s landscape into four categories. Knowing the difference helps you target your learning, resume, and projects.Role TypeWhat You Actually DoHow AI Affects ItProduct Data ScientistA/B testing, causal inference, experimentationStill high in demand. Requires sound statistical reasoning.Applied Data ScientistBuild customer-facing models and deploy pipelinesCoding and model tuning remain essential. Not heavily automated yet.Research Data ScientistCreate new model architectures and algorithmsLeast affected by automation. Requires deep ML expertise.AI Engineeroptimize LLM pipelines, work with vector DBs, prompts, deployment. Growing rapidly as companies scale GenAI.Inside Walmart, Karun shared that they already integrate ChatGPT-powered workflows internally. But even with AI tools in the loop, human oversight, system thinking, and engineering maturity remain essential.Python vs R in 2025: The Debate Is OverKarun did not hesitate here.Python has won.Why:It integrates with big data tools like PySparkIt is production friendlyIt is the default language in ML teamsIt has unmatched community supportR is still relevant for specialized statistical work, but new entrants are better off starting with Python. For performance heavy systems, C and C++ still matter, especially for latency sensitive pipelines.Want Recruiters To Notice You? Build Business Oriented ProjectsKarun’s portfolio advice is refreshingly practical.Stop doing random datasets. Start doing business problems.He recommends:Kaggle competitions with prize moneyRetail and financial datasets like M5 forecasting, H&M recommendations, Jane Street market predictionProjects where you quantify your performance against leadersOne golden rule:Always show how close you are to rank one. Not just your accuracy.👉 Continue reading the full article on the Packt Medium handle.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Python workflow guidance for analytics teams plus details on our upcoming algo trading session.Subscribe|Submit a tip|Advertise with Us📊 Welcome to BIPro 119This week we are bringing something new to our BI and analytics readers. We have launched live AMA sessions with industry experts on the Packt DataML YouTube channel, hosted by our Growth Lead Abhishek Kaushik. These sessions give BI practitioners direct access to practical guidance, workflow clarity, and real world perspectives from experts shaping the future of Python, AI, and data driven decision making. In this edition, we highlight insights from our latest conversation with Juan Gabriel Salas, one of Udemy’s highest rated Python and data science instructors. Juan shared actionable guidance on learning Python with purpose, connecting fundamentals to real analytical workflows, strengthening data intuition, and building work that stands out in a competitive BI landscape. You’ll find the full recap below.Knowledge Partner Spotlight: OutskillAt Packt, we’ve partnered withOutskillto help readers gain practical exposure to AI tools through free workshops, complementing the deeper, hands-on, expert-led experiences offered throughPackt Virtual Conferences.Become an AI Genius in One WeekendAs part of their Black Friday offer, Outskill is opening access to their 2 day LIVE AI Mastermind for free. This 16 hour intensive program covers AI tools, automations, and agent building, and is rated 9.8 on Trustpilot. It is usually priced at $395, but DataPro readers can join at no cost while seats last. The live sessions run Saturday and Sunday from 10 AM EST to 7 PM EST.Save your spot now, limited seats available.💡 Workshop Spotlight: Algorithmic Trading with Python (Cohort 2)Join Jason Strimpel for a hands-on workshop on December 12, where you’ll design, test, and deploy algorithmic trading strategies in Python using pandas, VectorBT, and the Interactive Brokers API. Whether you’re a quant-in-training or a finance pro, this live session takes you through the entire workflow, from identifying trading edges to executing real-world strategies.Pick a Full Pass or VIP Pass, use EARLY20 for 20% off, and reserve your seat.Register Now and Save With the Early Bird OfferCheers,Merlyn ShelleyGrowth Lead, PacktMastering Python and Data Science: Insights From a Live AMA WithJuan Gabriel SalasWith an aim to improve the reader experience, we are rolling out a new AMA series in partnership with our Expert Network to bring firsthand insights directly from industry leaders in the data world.Abhishek, our Growth Lead for the data science vertical, is hosting these live AMA sessions on thePackt DataML YouTube channelto help learners gain clarity, confidence, and direction in their data journey.This week’s session featuresJuan Gabriel Salas, a top rated Udemy instructor and founder of Frogames Formación. Juan is known for his practical teaching style, clear explanations, and deep experience helping thousands of learners master Python and build data science careers. Over the course of an hour, he shared a wide range of actionable insights on how to learn Python effectively, build strong intuition, create impactful portfolios, and navigate the fast evolving world of data science and machine learning.This article brings together all those insights in a single narrative so that every data learner can benefit from the clarity and wisdom that came out of the session.The Big Themes of the SessionPython fundamentals. Data workflows. Portfolio strategy. Math intuition. Future skills. And everything in between.To make this recap easier to navigate, we have organized the insights into six major themes that emerged during the AMA.1. Where Should Beginners Start: Python or Data Science?The conversation kicked off with a classic dilemma. Should learners start with Python basics or jump straight into data science?Juan’s advice was simple and rooted in experience.Start with Python, but not in a way that traps you in endless syntax drills. Instead, learn Python with a purpose. Focus on the parts of the language that matter for data work.You do not need to become a software engineer before becoming a data practitioner. But you do need enough fluency to manipulate data, write reusable code, and understand how your tools work behind the scenes.Python is not just another skill. It is the language that holds your entire workflow together.What matters most at the start:Knowing how to work with lists and dictionariesReading and writing filesUnderstanding functions and basic logicGetting comfortable with problem solvingOnce these fundamentals feel natural, transitioning into data science becomes far more enjoyable.2. Closing the Gap Between Syntax and Real Data Science WorkMany learners understand Python basics but freeze when facing a blank project. They know what a loop is but not how to apply one in a data pipeline. They understand lists but not how to use them in a transformation.Juan explained that this is not a lack of skill. It is a mindset shift that has not yet fully developed.Data science is less about writing isolated lines of code and more about seeing the bigger picture. It requires thinking insystems and workflowsrather than steps.You move from:“How do I write this line of code?”to“How do I design a process that answers this question?”You start seeing your work as a sequence of stages:Load. Clean. Explore. Transform. Model. Evaluate. Communicate.This shift is what turns a coder into a data practitioner.3. How to Approach a Real Data Science ProblemOne of the most valuable parts of the AMA was Juan’s breakdown of what to do when facing a new data science project from scratch. Instead of jumping into libraries or algorithms, he recommended focusing on three checkpoints that define successful work.Checkpoint 1: Understand Data QualityBefore choosing a model or writing transformations, you must examine the data. Learn to recognise missing values, duplicated rows, strange outliers, or mislabeled categories. This is where the bulk of practical skill is formed.Checkpoint 2: Choose the Right Modeling ApproachIs this a classification problem? A regression? A time series forecast? A clustering experiment?Knowing what problem you are solving is more important than the model itself.Checkpoint 3: Define Evaluation Metrics EarlyAccuracy is not always the right metric. Sometimes you need precision, recall, AUC, F1, RMSE, or business specific KPIs.Thinking about metrics early forces clarity.Together, these checkpoints create a stable foundation for any analysis.4. How Tiny Python Mistakes Can Break Entire ML WorkflowsJuan shared a humorous yet painful story about how a single indexing mistake caused inconsistencies across an entire machine learning pipeline. The model trained successfully. The predictions looked plausible. But the results made no sense.The culprit was a tiny transform that shifted rows.This resonated deeply with the audience because it illustrates a truth that every data practitioner eventually learns.Machine learning magnifies small mistakes.One misaligned array. One swapped column. One incorrect filter. One wrong assumption during data cleaning.This is why debugging is one of the most underrated data science skills. Being patient, methodical, and curious makes you far more effective than knowing every library function.5. What Makes a Portfolio Stand Out to EmployersPerhaps the most practical part of the AMA was Juan’s guidance on portfolio building. Many learners build beautiful notebooks that fail to impress employers simply because they focus on the wrong things.Juan broke down the three elements that make a portfolio truly meaningful.1. Use Real World DataAvoid overused toy datasets. Instead, explore domains you care about. Scrape data. Gather logs. Use messy CSVs. These reveal how you work, not how well you follow tutorials.2. Show Your ThinkingEmployers want to know why you chose an approach, what decisions you made, and how you debugged problems. Narrative clarity matters just as much as code.3. Focus on Depth, Not QuantityFive shallow projects will not beat two strong, thoughtful, well documented projects with clear reasoning and solid results.A strong portfolio is a story, not a showcase.👉 Continue reading the full article on the Packt Medium handle.See you next time!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}