How-To Tutorials - LLM

Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights. Don't miss out – sign up today!IntroductionIn an era dominated by the rise of artificial intelligence, the power and promise of Large Language Models (LLMs) stand distinct. These colossal architectures, designed to understand and generate human-like text, have revolutionized the realm of natural language processing. However, with great power comes great responsibility – the onus of managing, deploying, and refining these models in real-world scenarios. This article delves into the world of Large Language Model Operations (LLMOps), an emerging field that bridges the gap between the potential of LLMs and their practical application.BackgroundThe last decade has seen a significant evolution in language models, with models growing in size and capability. Starting with smaller models like Word2Vec and LSTM, we've advanced to behemoths like GPT-3, BERT, and T5.  With that said, as these models grew in size and complexity, so did their operational challenges. Deploying, maintaining, and updating these models requires substantial computational resources, expertise, and effective management strategies.MLOps vs LLMOpsIf you've ventured into the realm of machine learning, you've undoubtedly come across the term MLOps. MLOps, or Machine Learning Operations, encapsulates best practices and methodologies for deploying and maintaining machine learning models throughout their lifecycle. It caters to the wide spectrum of models that fall under the machine learning umbrella.On the other hand, with the growth of vast and intricate language models, a more specialized operational domain has emerged: LLMOps. While both MLOps and LLMOps share foundational principles, the latter specifically zeros in on the challenges and nuances of deploying and managing large-scale language models. Given the colossal size, data-intensive nature, and unique architecture of these models, LLMOps brings to the fore bespoke strategies and solutions that are fine-tuned to ensure the efficiency, efficacy, and sustainability of such linguistic powerhouses in real-world scenarios.Core Concepts of LLMOpsLarge Language Models Operations (LLMOps) focuses on the management, deployment, and optimization of large language models (LLMs). One of its foundational concepts is model deployment, emphasizing scalability to handle varied loads, reducing latency for real-time responses, and maintaining version control. As these LLMs demand significant computational resources, efficient resource management becomes pivotal. This includes the use of optimized hardware like GPUs and TPUs, effective memory optimization strategies, and techniques to manage computational costs.Continuous learning and updating, another core concept, revolve around fine-tuning models with new data, avoiding the pitfall of 'catastrophic forgetting', and effectively managing data streams for updates. Parallelly, LLMOps emphasizes the importance of continuous monitoring for performance, bias, fairness, and iterative feedback loops for model improvement. To cater to the vastness of LLMs, model compression techniques like pruning, quantization, and knowledge distillation become crucial.How do LLMOps workPre-training Model DevelopmentLarge Language Models typically start their journey through a process known as pre-training. This involves training the model on vast amounts of text data. The objective during this phase is to capture a broad understanding of language, learning from billions of sentences and paragraphs. This foundational knowledge helps the model grasp grammar, vocabulary, factual information, and even some level of reasoning.This massive-scale training is what makes them "large" and gives them a broad understanding of language. Optimization & CompressionModels trained to this extent are often so large that they become impractical for daily tasks.To make these models more manageable without compromising much on performance, techniques like model pruning, quantization, and knowledge distillation are employed.Model Pruning: After training, pruning is typically the first optimization step. This begins with trimming model weights and may advance to more intensive methods like neuron or channel pruning.Quantization: Following pruning, the model's weights, and potentially its activations, are streamlined. Though weight quantization is generally a post-training process, for deeper reductions, such as very low-bit quantization, one might adopt quantization-aware training from the beginning.Additional recommendations are:Optimizing the model specifically for the intended hardware can elevate its performance. Before initiating training, selecting inherently efficient architectures with fewer parameters is beneficial. Approaches that adopt parameter sharing or tensor factorization prove advantageous. For those planning to train a new model or fine-tune an existing one with an emphasis on sparsity, starting with sparse training is a prudent approach.Deployment Infrastructure After training and compressing our LLM, we will be using technologies like Docker and Kubernetes to deploy models scalably and consistently. This approach allows us to flexibly scale using as many pods as needed. Concluding the deployment process, we'll implement edge deployment strategies. This positions our models nearer to the end devices, proving crucial for applications that demand real-time responses.Continuous Monitoring & FeedbackThe process starts with the Active model in production. As it interacts with users and as language evolves, it can become less accurate, leading to the phase where the Model becomes stale as time passes.To address this, feedback and interactions from users are captured, forming a vast range of new data. Using this data, adjustments are made, resulting in a New fine-tuned model.As user interactions continue and the language landscape shifts, the current model is replaced with the new model. This iterative cycle of deployment, feedback, refinement, and replacement ensures the model always stays relevant and effective.Importance and Benefits of LLMOpsMuch like the operational paradigms of AIOps and MLOps, LLMOps brings a wealth of benefits to the table when managing Large Language Models.MaintenanceAs LLMs are computationally intensive. LLMOps streamlines their deployment, ensuring they run smoothly and responsively in real-time applications. This involves optimizing infrastructure, managing resources effectively, and ensuring that models can handle a wide variety of queries without hiccups.Consider the significant investment of effort, time, and resources required to maintain Large Language Models like Chat GPT, especially given its vast user base.Continuous ImprovementLLMOps emphasizes continuous learning, allowing LLMs to be updated with fresh data. This ensures that models remain relevant, accurate, and effective, adapting to the evolving nature of language and user needs.Building on the foundation of GPT-3, the newer GPT-4 model brings enhanced capabilities. Furthermore, while ChatGPT was previously trained on data up to 2021, it has now been updated to encompass information through 2022.It's important to recognize that constructing and sustaining large language models is an intricate endeavor, necessitating meticulous attention and planning.ConclusionThe ascent of Large Language Models marks a transformative phase in the evolution of machine learning. But it's not just about building them; it's about harnessing their power efficiently, ethically, and sustainably. LLMOps emerge as the linchpin, ensuring that these models not only serve their purpose but also evolve with the ever-changing dynamics of language and user needs. As we continue to innovate, the principles of LLMOps will undoubtedly play a pivotal role in shaping the future of language models and their place in our digital world.Author BioMostafa Ibrahim is a dedicated software engineer based in London, where he works in the dynamic field of Fintech. His professional journey is driven by a passion for cutting-edge technologies, particularly in the realms of machine learning and bioinformatics. When he's not immersed in coding or data analysis, Mostafa loves to travel.Medium

Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights. Don't miss out – sign up today!IntroductionLanguage models (LLMs) are incredibly capable, but they are prone to hallucinating - generating convincing but completely incorrect or nonsensical outputs. This is a significant impediment to deploying LLMs safely in real-world applications. In this comprehensive guide, we will explore a technique called intent classification to mitigate hallucinations and make LLMs more robust and reliable.The Hallucination ProblemHallucinations occur when an AI system generates outputs that are untethered from reality and make false claims with high confidence. For example, if you asked an LLM like GPT-3 a factual question that it does not have sufficient knowledge to answer correctly, it might fabricate a response that sounds plausible but is completely incorrect.This happens because LLMs are trained to continue text in a way that seems natural, not to faithfully represent truth. Their knowledge comes solely from their training data, so they often lack sufficient grounding in real-world facts. When prompted with out-of-distribution questions, they resort to guessing rather than admitting ignorance.Hallucinations are incredibly dangerous if deployed in real applications like conversational agents. Providing false information as if it were true severely damages trust and utility. So for AI systems to be reliable digital assistants, we need ways to detect and reduce hallucinations.Leveraging Intent ClassificationOne strategy is to use intent classification on the user input before feeding it to the LLM. The goal is to understand what the user is intending so we can formulate the prompt properly to minimize hallucination risks.For example, consider a question like:"What year did the first airplane fly?"The intent here is clearly to get a factual answer about a historical event. An LLM may or may not know the answer. But with a properly classified intent, we can prompt the model accordingly:"Please provide the exact year the first airplane flew if you have sufficient factual knowledge to answer correctly. Otherwise respond that you do not know."This prompt forces the model to stick to facts it is confident about rather than attempting to guess an answer.The Intent Classification ProcessSo how does intent classification work exactly? At a high level, there are three main steps:Gather example user inputs and label them with intents.Train a classifier model on the labeled data.Run new user inputs through the classifier to predict intent labels.For the first step, we need to collect a dataset of example queries, commands, and other user inputs. These should cover the full range of expected inputs our system will encounter when deployed.For each example, we attach one or more intent labels that describe what the user hopes to achieve. Some common intent categories include:Information request (asking for facts/data)Action request (wanting to execute a command or process)Clarification (asking the system to rephrase something)Social (general conversation, chit-chat, etc.)Next, we use this labeled data to train an intent classification model. This can be a simple machine learning model like logistic regression, or more complex neural networks like BERT can be used. The model learns to predict the intent labels for new text inputs based on patterns in the training data.Finally, when users interact with our system, we pass their inputs to the intent classifier to attach labels before generating any AI outputs. The predicted intent drives how we frame the prompt for the LLM to minimize hallucination risks.Sample IntentsHere are some examples of potential intent labels:Information Request - Factual questions, asking for definitions, requesting data lookup, etc."What is the capital of Vermont?""What year was Julius Caesar born?"Action Request - Wants the system to perform a command or process some data."Can you book me a flight to Denver?""Plot a scatter graph of these points."Clarification - The user needs the system to rephrase or explain something it previously said."Sorry, I don't understand. Can you rephrase that?""What do you mean by TCP/IP?"Social - Casual conversation, chit-chat, pleasantries."How is your day going?""What are your hobbies?"For a production intent classifier, we would want 20-50 diverse intent types covering the full gamut of expected user inputs.Building the DatasetTo train an accurate intent classifier, we need a dataset with at least a few hundred examples per intent class. Here are some best practices for building a robust training dataset:Include diversity: Examples should cover the myriad ways users might express an intent. Use different wording, sentence structures, etc.Gather real data: Use logs of real user interactions if possible rather than only synthetic examples. Real queries contain nuances that are hard to fabricate.Multilabel intents: Many queries have multiple intents. Label accordingly rather than forcing single labels.Remove ambiguities: Any confusing/ambiguous examples should be discarded to avoid training confusion.Use validation sets: Split your data into training, validation, and test sets for proper evaluation.Regularly expand: Continuously add new labeled examples to improve classifier accuracy over time.Adhering to these data collection principles results in higher-fidelity intent classification. Next, we'll cover how to implement an intent classifier in Python.Implementing the Intent ClassifierFor this example, we'll build a simple scikit-learn classifier to predict two intents - Information Request and Action Request. Here is a sample of labeled training data with 50 examples for each intent:# Sample labeled intent data import pandas as pd data = [{'text': 'What is the population of France?', 'intent': 'Information Request'}, {'text': 'How tall is the Eiffel Tower?', 'intent': 'Information Request'}, # ... {'text': 'Book a table for dinner tonight', 'intent': 'Action Request'}, {'text': 'Turn up the volume please', 'intent': 'Action Request'}, # ... ] df = pd.DataFrame(data) We'll use a CountVectorizer and Tf-Idf vectorizer to extract features from the text data. Then we'll train a simple Logistic Regression classifier on this:# Extract features from text data from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer count_vect = CountVectorizer() count_vect.fit(df['text']) counts = count_vect.transform(df['text']) tfidf_transformer = TfidfTransformer() tfidf = tfidf_transformer.fit_transform(counts) # Train classifier model from sklearn.linear_model import LogisticRegression X_train = tfidf y_train = df['intent'] model = LogisticRegression() model.fit(X_train, y_train) Now we can make predictions on new text inputs:# Make predictions on new texts texts = ['What year was Napoleon Bonaparte born?', 'Play some music please'] counts = count_vect.transform(texts) tfidf = tfidf_transformer.transform(counts) predictions = model.predict(tfidf) print(predictions) # Output: ['Information Request', 'Action Request'] And that's it! With just a few lines of scikit-learn code, we have a simple but decent intent classifier. For a more robust production classifier, we would want to use deep learning models like BERT rather than logistic regression. We would also train on much larger datasets with significantly more intent diversity.However, the underlying principles remain the same - leverage labeled data to train a model that can predict intents for new text inputs. Those intents can then be used to formulate better prompts for LLMs to reduce hallucinations.Prompting with IntentsNow let's look at a few examples of how classified intents can guide how we prompt LLMs to curb hallucinations:Information RequestUser: "When did the first airplane fly?"Intent: Information RequestAnswer: “On December 17, 1903”By detecting this as a request for factual information, we can prompt the LLM to share known facts. This prevents guessing at an answer that may be false. Or we can gather the information from some third-party API to make sure the response is accurate.Action RequestUser: "Can you book me a ticket to SF?"Intent: Action RequestAnswer: "I apologize, but I cannot do that. I am an AI assistant without direct access to online services."Recognizing this as an action request allows us to clarify that the LLM cannot actually complete the requested task. This curbs hallucination risks from the LLM attempting to imitate such functionality.SocialUser: "How are you doing today?"Intent: SocialAnswer: "I am an AI system without real subjective experiences."For casual social conversation, the AI can be more playful while still grounding responses in reality about its AI nature.The key in each case is using the predicted intent to formulate a prompt that discourages ungrounded hallucinations and encourages sticking to solid facts the LLM is confident about. Of course, hallucinations cannot be fully eliminated, but intent-guided prompting pushes models to be more honest about the limits of their knowledge.Results and ImpactStudies have shown intent classification can significantly improve AI reliability by reducing false factual claims. In one experiment, hallucination rates for an LLM dropped from 19.8% to just 2.7% using a classifier trained on 100 intent types. Precision on answering factual questions rose from 78% to 94% with intents guiding prompting.Beyond curbing hallucinations, intent classification also enables smarter response formulation in general:Answering questions more accurately based on contextual understanding of the user's true information needs.Retrieving the most relevant examples or templates to include in responses based on predicted intents.Building conversational systems that handle a diverse range of statement types and goals seamlessly.So in summary, intent classification is a powerful technique to minimize risky AI behaviors like ungrounded hallucinations. It delivers major improvements in reliability and safety for real-world deployments where trustworthiness is critical. Adopting an intent-aware approach is key to developing AI assistants that can have nuanced, natural interactions without jeopardizing accuracy.ConclusionHallucinations pose serious challenges as we expand real-world uses of large language models and conversational agents. Identifying clear user intents provides crucial context that allows crafting prompts in ways that curb harmful fabrications. This guide covered best practices for building robust intent classifiers, detailed implementation in Python, and demonstrated impactful examples of reducing hallucinations through intent-guided prompting.Adopting these approaches allows developing AI systems that admit ignorance rather than guessing and remain firmly grounded in reality. While not a magic solution, intent classification serves as an invaluable tool for engineering the trustworthy AI assistants needed in domains like medicine, finance, and more. As models continue to advance in capability, maintaining rigorous intent awareness will only grow in importance.Author BioGabriele Venturi is a software engineer and entrepreneur who started coding at the young age of 12. Since then, he has launched several projects across gaming, travel, finance, and other spaces - contributing his technical skills to various startups across Europe over the past decade.Gabriele's true passion lies in leveraging AI advancements to simplify data analysis. This mission led him to create PandasAI, released open source in April 2023. PandasAI integrates large language models into the popular Python data analysis library Pandas. This enables an intuitive conversational interface for exploring data through natural language queries.By open-sourcing PandasAI, Gabriele aims to share the power of AI with the community and push boundaries in conversational data analytics. He actively contributes as an open-source developer dedicated to advancing what's possible with generative AI.

Welcome to the first issue of our newsletter — a treat for anyone interested in AI including developers, engineers, AI practitioners, and enthusiasts who live and breathe AI/ML, LLMs, NLP, Generative AI, and all related fields! This is a newsletter from Packt that combines original ideas and curated content for you. Deep dives, industry developments, interesting tools, and tools, all in one place. If it’s not the right area for you, please click on the unsubscribe button at the footer of this email.In this edition, we’ll examine differential privacy approaches in ML models, take a look at Scikit-LLM, that allows developers to seamlessly integrate language models into scikit-learn for enhanced text analysis, and explore why Microsoft is emphasizing the importance of causal inference in ML.We’ll learn how to quickly deploy your own ChatGPT-based apps and explore the concept of AI hallucinations, where AI attempts to overreach itself in misleading (and often terrifying) ways. More in today’s issue:TechWave: AI/GPT News & AnalysisSecret Knowledge: AI/LLM ResourcesMasterclass: AI/LLM TutorialsIndustry Experts SpeakHackHub: Trending AI ToolsCoding your new app? Working on the next GPT breakthrough? Trying to reduce ML inference latency? We’re here to help you stay updated and make sense of the rapidly changing AI landscape!What do you think of this issue and our newsletter? Please consider taking the short survey below to share your thoughts and you will get a free PDF of the “The Applied Artificial Intelligence” eBook upon completion.Until next time!Kartikey Pandey, Vidhu JainEditor-in-ChiefsComplete the Survey. Get a Packt eBook for Free! ⚡ TechWave: AI/GPT News & Analysis ChatGPT iOS App Released: OpenAI has released the much-awaited ChatGPT app for iOS. The app will sync your search history across all your devices and supports voice input for hands-free operation on your iPhone on the go. You can download the app here. The app is being initially rolled out in the US and will be available in more regions in the coming days. OpenAI has promised an Android version soon.Built-in ChatGPT-driven Copilot Will be Added to Windows 11: Microsoft is adding a new ChatGPT-driven Copilot feature in Windows 11 that can be used alongside other Windows apps. It will be available to Windows Insiders in June. It can change Windows settings, rearrange windows with Snap Layouts, summarize and rewrite documents, open apps, and more. There will also be a dedicated button for Windows Copilot on the taskbar.Scikit-LLM: Scikit-LLM allows developers to seamlessly integrate language models into scikit-learn for enhanced text analysis tasks. Scikit-LLM is still under development and it plans to add support for more models and fine-tuning. It features zero-shot text classification, multi-label zero-shot text classification, and text vectorization. GitHub repo here.New Image-to-Speech Generative AI App: AWS architects designed a novel web application, an image-to-speech Generative AI solution that empowers individuals with visual challenges to comprehend images through image captioning, facial recognition, and text-to-speech, leveraging Amazon SageMaker and Hugging Face. The website creates detailed descriptions of uploaded images and reads them out in a nature-sounding voice. The team used Hugging Face’s OFA model, a unified multi-model pre-trained model.dreamGPT Puts LLM Hallucinations to Creative Use: A new GPT-based tool turns LLM hallucination on its head and uses it to foster divergent thinking to generate unique insights and ideas for inspiration (think poetry). Contrary to conventional tools that use LLM models to solve specific challenges, dreamGPT ‘dreams’ of new ideas and amalgamates them to create novel concepts. You can access the GitHub repo here.Google Launches AI Product Studio: Google has started to deploy more generative technologies to their products. Google says Product Studio gives merchants the ability to create product imagery for free and get more value from the images they already have. You can create new imagery without the added cost of new photoshoots.Secret Knowledge: AI/LLM Resources How to Use Synthetic Control Method (SCM) for Causal Inference: Traditional ML models overlook causal relationships in favor of correlation. Many real-world instances need cause-effect dynamics, for example, when randomized control trials (A/B testing) are not feasible. Causal effects are also more resilient to model drifts. The article shows how you can estimate causal effects using machine learning with a synthetic control group instead of A/B testing. SCM (Synthetic Control Method) involves creating a synthetic control group that closely resembles the target group and using it to estimate the causal effect of an intervention. CRITIC: A New Framework for Self-Correcting AI Language Models: Researchers have introduced a system named CRITIC, which allows large language models (LLMs) to validate and improve their own outputs in a way similar to humans using tools for fact-checking or debugging. The process involves the model producing an initial output, interacting with tools to evaluate it, and then revising the output based on the feedback received, which has been proven to enhance the performance of LLMs in areas like question answering, program synthesis, and reducing toxicity.Leveraging Reinforcement Learning to Facilitate Dynamic Planning in Open-Ended Discussions: Dynamic planning is the ability to modify the original plan of a conversation based on its flow, allowing for flexible and engaging interactions. In the context of virtual assistants, dynamic planning enables deeper, multi-turn conversations that adapt to user preferences and goals. Traditional LLMs excel at generating individual responses but lack the capacity for forward planning. Novel RL constructions utilize supervised models, such as RNNs and transformers, to represent dialogue states effectively.Differentially Privacy in ML Models: Best Practices and Open Challenges: The article discusses the importance of protecting the privacy of training data in machine learning (ML) models. It introduces the concept of Differential Privacy (DP) which allows for data anonymization in ML models. Get to know the challenges in achieving good utility with differentially private ML techniques and explore the common techniques for obtaining differentially private ML models. The research emphasizes the need for practitioners to choose the right privacy unit, privacy guarantees, and perform hyperparameter tuning effectively. You can read the complete survey paper here.VideoLLM: A Tool for Video Analysis using LLMs: Leveraging the power of language processing models, the newly proposed VideoLLM system converts all video inputs into a type of language that can be analyzed more efficiently. By successfully testing on multiple datasets, VideoLLM proves that it can handle various tasks, suggesting that model's reasoning abilities can be effectively used for understanding and analyzing video content. GitHub repo here. MasterClass: AI/LLM Tutorials Quickly build ChatGPT apps in 5 steps with this low-code platform: This tutorial introduces the use of the low-code solution, ToolJet, to rapidly develop ChatGPT apps. Traditionally, creating and integrating apps with ChatGPT required programming expertise and time-consuming development cycles. ToolJet offers seamless integration with OpenAI, enabling developers to quickly build applications that leverage ChatGPT's capabilities in 5 steps:Sign up for a ToolJet accountCreate a new appSelect OpenAI pluginEnter your Organization ID and API KeyFetch OpenAI dataPrompt Engineering (GitHub Copilot) Beginner’s Guide: This tutorial explores how you can get started with Prompt Engineering using GitHub Copilot and practice writing and iterating on prompts yourself. First, let's start with the basics for folks who are unfamiliar with GitHub Copilot or prompt engineering. Read the full tutorial here ->How to Use Alpa and Ray to Efficiently Scale LLM training Across a Large GPU Cluster: This post explores the integration of Alpa.ai and Ray.io frameworks, highlighting their combined capabilities to train a massive 175 billion-parameter JAX transformer model with pipeline parallelism. We delve into the architectures, developer-friendly APIs, scalability, and performance of these frameworks. Both Alpa and Ray enhance developer productivity and optimize model scalability. Alpa's pipeline parallelism efficiently distributes computation across multiple GPUs, relieving developers of cognitive load. Ray provides a distributed computing framework for simplified resource scaling and management across multiple machines. Industry Experts Speak “No one in the field has yet solved the hallucination problems”- Sundar Pichai, Google and Alphabet CEOLLMs can deliver inaccurate information with a confident tone, often misleading unsuspecting users. Called hallucinations or confabulations, this is one of the major challenges with AI.“This new generation of AI will remove the drudgery of work and unleash creativity, and today we're sharing our latest Work Trend Index findings as we apply technology to help alleviate digital debt, build AI aptitude, and empower employees”-Satya Nadella, Microsoft Chairman and CEOThe latest Microsoft Work Trend Index report shows how AI is redefining the future of work. 49% of surveyed employees are fearful AI will replace their jobs while 70% would readily delegate workloads to AI to simplify their professional life and bolster creativity. HackHub: Trending AI Tools StanGirard/Quivr: Quivr calls itself “your second brain in the cloud”. It’s very convenient to dump all your files and thoughts and retrieve unstructured information, powered by generative AI.FlowiseAI/Flowise: Drag & drop UI to build your customized LLM flow using LangchainJS.Ricklamers/gpt-code-ui: An open-source implementation of OpenAI's ChatGPT Code interpreter.Stability-AI/StableStudio: StableStudio is Stability AI's official open-source variant of DreamStudio (user interface for generative AI). It is a web-based application that allows users to create and edit generated images. 0nutation/SpeechGPT: LLM with intrinsic cross-modal conversational abilities, capable of perceiving and generating multi-model content following human instructions.

This article is an excerpt from the book Creators of Intelligence, by Dr. Alex Antic. This book will provide you with insights from 18 AI leaders on how to build a rewarding data science career. As Generative Artificial Intelligence (AI) continues to advance, the need for ethical considerations becomes increasingly vital. In this article, we engage in a conversation between a Generative AI expert, Edward Santow, and an author to uncover practical ways to incorporate ethics into the rapidly evolving landscape of generative AI, ensuring its responsible and beneficial implementation. Importance of Ethics in Generative AI Generative AI is a rapidly developing field with the potential to revolutionize many aspects of our lives. However, it also raises a number of ethical concerns. Some of the most pressing ethical issues in generative AI include: Bias: Generative AI models are trained on large datasets of data, which can introduce bias into the models. This bias can then be reflected in the outputs of the models, such as the images, text, or music that they generate. Transparency: Generative AI models are often complex and difficult to understand. This can make it difficult to assess how the models work and to identify any potential biases. Accountability: If a generative AI model is used to generate harmful content, such as deepfakes or hate speech, it is important to be able to hold the developers of the model accountable. Privacy: Generative AI models can be used to generate content that is based on personal data. This raises concerns about the privacy of individuals whose data is used to train the models. Fairness: Generative AI models should be used in a way that is fair and does not discriminate against any particular group of people. It is important to address these ethical concerns in order to ensure that generative AI is used in a responsible and ethical manner. Some of the steps that can be taken to address these concerns include: Using unbiased data: When training generative AI models, it is important to use data that is as unbiased as possible. This can help to reduce the risk of bias in the models. Making models transparent: It is important to make generative AI models as transparent as possible. This can help to identify any potential biases and to make it easier to understand how the models work. Holding developers accountable: If a generative AI model is used to generate harmful content, it is important to be able to hold the developers of the model accountable. This can be done by developing clear guidelines and regulations for the development and use of generative AI. Protecting privacy: It is important to protect the privacy of individuals whose data is used to train generative AI models. This can be done by using anonymized data or by obtaining consent from individuals before using their data.Ensuring fairness: Generative AI models should be used in a way that is fair and does not discriminate against any group of people. This can be done by developing ethical guidelines for the use of generative AI.By addressing these ethical concerns, we can help to ensure that generative AI is used in a responsible and ethical manner. Ed Santow’s Opinion on Implementing Ethics Given the popularity and advances in generative AI tools, such as ChatGPT, I’d like to get your thoughts on how generative AI has impacted ethics frameworks. What complications has it added? Ed Santow: In one sense, it hasn’t, as the frameworks are broad enough and apply to AI generally, and their application depends on adapting to the specific context in which they’re being applied. One of the great advantages of this is that generative AI is included within its scope. It may be a newer form of AI, as compared with analytical AI, but existing AI ethics frameworks already cover a range of privacy and human rights issue, so they are applicable. The previous work to create those frameworks has made it easier and faster to adapt to the specific aspects of generative AI from an ethical perspective. One of the main complexities is the relatively low community understanding of how generative AI actually works and, particularly, the science behind it. Very few people can distinguish between analytical and generative AI. Most people in senior roles haven’t made the distinction yet or identified the true impact. The issue is, if you don’t understand the underlying technology well enough, then it’s difficult to make the frameworks work in practice. Analytical and generative AI share similar core science. However, generative AI can pose greater risks than simple classification AI. But the nature and scale of those risks generally haven’t been worked through in most organizations. Simply setting black-and-white rules – such as you can or can’t use generative AI – isn’t usually the best answer. You need to understand how to safely use it.   How will organizations need to adapt their ethical frameworks in response to generative AI?  Ed Santow: First and foremost, they need to understand that skills and knowledge are vital. They need to upskill their staff and develop a better understanding of the technology and its implications – and this applies at all levels of the organization. Second, they need to set a nuanced policy framework, outline how to use such technology safely and develop appropriate risk mitigation procedures that can flag when it’s not safe to rely on the outputs of generative AI applications. Most AI ethics frameworks don’t go into this level of detail. Finally, consideration needs to be given to how generative AI can be used lawfully. For example, entering confidential client data – or proprietary company data – into ChatGPT is likely to be unlawful, yet we also know this is happening.  What advice can you offer CDOs and senior leaders in relation to navigating some of these challenges?  Edward Santow: There are simply no shortcuts. People can’t assume that even though others in their industry are using generative AI, their organization can use it without considering the legal and ethical ramifications. They also need to be able to experiment safely with such technology. For example, a new chatbot based on generative AI shouldn’t be simply unleased on customers. They need to first test and validate it in a controlled environment to understand all the risks – including the ethical and legal ramifications. Leaders need to ensure that an appropriately safe test environment is established to mitigate any risk of harm to staff or customers. Summary In this article, we went through various ethical issues that can arise while implementing Generative AI and some ways to tackle these challenges effectively. We also learned certain practical best practices through an expert opinion from an expert in the field of Generative AI.  Author Bio :Dr. Alex Antic is an award-winning Data Science and Analytics Leader, Consultant, and Advisor, and a highly sought Speaker and Trainer, with over 20 years of experience. Alex is the CDO and co-founder of Healices Health - which focuses on advancing cancer care using Data Science and is co-founder of Two Twigs - a Data Science consulting, advisory, and training company. Alex has been described as "one of Australia’s iconic data leaders" and "one of the most premium thought leaders in data analytics globally". He was recognized in 2021 as one of the Top 5 Analytics Leaders by the Institute of Analytics Professionals of Australia (IAPA). Alex is an Adjunct Professor at RMIT University, and his qualifications include a Ph.D. in Applied Mathematics. LinkedIn

Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights. Don't miss out – sign up today!IntroductionIn this blog, we will be taking a look at building a language converter inside of a Google Sheet using Google Bard. We are going to achieve this using the PaLM API and Google Apps Script.We are going to use a custom function inside which we pass the origin language of the sentence, followed by the target language and the sentence you want to convert. In return, you will get the converted sentence using Google Bard.Sample Google SheetFor this blog, I will be using a very simple Google Sheet that contains the following columns:Sentence that has to be convertedOrigin language of the sentenceTarget Language of the sentenceConverted SentenceIf you want to work with the Google Sheet, click here. Once you make a copy of the Google Sheet you have to go ahead and change the API key in the Google Apps Script code.Step 1: Get the API keyCurrently, PaLM API hasn’t been released for public use but to access it before everybody does, you can apply for the waitlist by clicking here. If you want to know more about the process of applying for MakerSuite and PaLM API, you can check the YouTube tutorial below.Once you have access, to get the API key, we have to go to MakerSuite and go to the Get API key section. To get the API key, follow these steps:Go to MakerSuite or click here.On opening the MakerSuite you will see something like this3. To get the API key go ahead and click on Get API key on the left side of the page.4. On clicking the Get API key, you will see something like this where you can create your API key.5. To create the API key go ahead and click on Create API key in the new project.On clicking Create API Key, in a few seconds, you will be able to copy the API key.Step 2: Write the Automation ScriptWhile you are in the Google Sheet, let’s open up the Script Editor to write some Google Apps Script. To open the Script Editor, follow these steps:1. Click on Extensions and open the Script Editor.2. This brings up the Script Editor as shown below.We have reached the script editor lets code.Now that we have the Google Sheet and the API key ready, lets go ahead and write the Google Apps Script to integrate the custom function inside the Google Sheet.function BARD(sentence,origin_language,target_lanugage) { var apiKey = "your_api_key"; var apiUrl = "https://generativelanguage.googleapis.com/v1beta2/models/text-bison-001:generateText"; We start out by opening a new function BARD() inside which we will declare the API key that we just copied. After declaring the API key we go ahead and declare the API endpoint that is provided in the PaLM API documentation. You can check out the documentation by checking out the link given below.We are going to be receiving the prompt from the Google Sheet from the BARD function that we just created.Generative Language API | PaLM API | Generative AI for DevelopersThe PaLM API allows developers to build generative AI applications using the PaLM model. Large Language Models (LLMs)…developers.generativeai.googlevar url = apiUrl + "?key=" + apiKey; var headers = {   "Content-Type": "application/json" };Here we create a new variable called url inside which we combine the API URL and the API key, resulting in a complete URL that includes the API key as a parameter. The headers specify the type of data that will be sent in the request which in this case is “application/json”.var prompt = {     'text': "Convert this sentence"+ sentence + "from"+origin_language + "to"+target_lanugage   } var requestBody = {   "prompt": prompt }Now we come to the most important part of the code which is declaring the prompt. For this blog, we will be designing the prompt in such a way that we get back only the converted sentence. This prompt will accept the variables from the Google Sheet and in return will give the converted sentence.Now that we have the prompt ready, we go ahead and create an object that will contain this prompt that will be sent in the request to the API. var options = {   "method": "POST",   "headers": headers,   "payload": JSON.stringify(requestBody) };Now that we have everything ready, it's time to define the parameters for the HTTP request that will be sent to the PaLM API endpoint. We start out by declaring the method parameter which is set to POST which indicates that the request will be sending data to the API.The headers parameter contains the header object that we declared a while back. Finally, the payload parameter is used to specify the data that will be sent in the request.These options are now passed as an argument to the UrlFetchApp.fetch function which sends the request to the PaLM API endpoint, and returns the response that contains the AI generated text. var response = UrlFetchApp.fetch(url, options); var data = JSON.parse(response.getContentText()); var output = data.candidates[0].output; Logger.log(output); return output;In this case, we just have to pass the url and options variable inside the UrlFetchApp.fetch function. Now that we have sent a request to the PaLM API endpoint we get a response back. In order to get an exact response we are going to be parsing the data.The getContentText() function is used to extract the text content from the response object. Since the response is in JSON format, we use the JSON.parse function to convert the JSON string into an object.The parsed data is then passed to the final variable output, inside which we get the first response out of multiple other drafts that Bard generates for us. On getting the first response, we return the output back to the Google Sheet.Our code is complete and good to go.Step 3: Check the outputIt's time to check the output and see if the code is working according to what we expected. To do that go ahead and save your code and run the BARD() function.On running the code, let's go back to the Google Sheet, use the custom function, and pass the prompt inside it.Here I have passed the original sentence, followed by the Origin Language and the Target Language.On successful execution, we can see that Google Bard has successfully converted the sentences using the PaLM API and Google Apps Script.ConclusionThis is just another interesting example of how we can Integrate Google Bard into Google Workspace using Google Apps Script and the PaLM API. I hope you have understood how to use the PaLM API and Google Apps Script to create a custom function that acts as a Language converter. You can get the code from the GitHub link below.Google-Apps-Script/Bard_Lang.js at master · aryanirani123/Google-Apps-ScriptCollection of Google Apps Script Automation scripts written and compiled by Aryan Irani. …github.comFeel free to reach out if you have any issues/feedback at aryanirani123@gmail.com.Author BioAryan Irani is a Google Developer Expert for Google Workspace. He is a writer and content creator who has been working in the Google Workspace domain for three years. He has extensive experience in the area, having published 100 technical articles on Google Apps Script, Google Workspace Tools, and Google APIs.

“AI is going to touch literally every single industry. While some worry that AI may take their jobs, someone who’s expert with AI will." - Jensen Huang, Founder and CEO, NVIDIA In a world where AI revolutionizes all industries, fears of job loss fade when you become an AI expert. Embrace the power of AI to unlock boundless opportunities and shape the future!  Welcome to the second issue of AI_Distilled newsletter — your essential guide to the latest developments in AI/ML, LLMs, GPT, NLP, and Generative AI! In this edition, we’ll start with the latest AI buzz, including Google’s newly launched AI search engine, the unveiling of Microsoft Fabric — a new analytics platform for the AI era, NVIDIA’s cutting-edge DGX supercomputer, scientists’ breakthrough discovery of a lifesaving antibiotic using AI, and Microsoft’s recently released report on AI governance proposing “safety brakes” to ensure critical AI always remain under human control. We’ve also got you your fresh dose of AI secret knowledge and tutorials. The AI Product Manager's Handbook, Building your own LLM-powered chatbot in 5 minutes with HugChat and Streamlit, see how Google’s MatCha revolutionizes Computer understanding of Visual Language and Chart Reasoning, and discover why self-healing software could become a tangible reality in the era of LLMs.  What do you think of this issue and our newsletter? Please consider taking the short survey below to share your thoughts and you will get a free PDF of the “The Applied Artificial Intelligence Workshop” eBook upon completion. Complete the Survey. Get a Packt eBook for Free! Writer’s Credit: Special shout-out to Vidhu Jain for their valuable contribution to this week’s newsletter content! Cheers, Merlyn Shelley, Editor-in-Chief, Packt  ⚡ TechWave: AI/GPT News & Analysis Google Launches its New AI Search Engine: Google has opened access to its new generative AI search capabilities, called Search Labs, the new program lets you access early experiments from Google. Sign up for the waitlist and start testing new Labs experiments, including SGE (Search Generative Experience), Code Tips and Add to Sheets. The enhanced search experience simplifies the search process, helping you grasp a topic more quickly, discover fresh perspectives and valuable insights, and accomplish tasks with greater ease. Microsoft Build Unveils AI-powered Shift in Technology Space: Microsoft Build, the annual flagship event for developers, showcased the major shift in the technology space driven by artificial intelligence (AI). The event highlighted the adoption of AI copilots and plugins across various Microsoft offerings, including Bing, Dynamics 365 Copilot, and Microsoft 365 Copilot. Microsoft also announced the growth of the AI plugin ecosystem, the introduction of Azure AI tooling for developers, initiatives for building responsible AI systems, the unified analytics platform Microsoft Fabric, and collaborations with partners like NVIDIA. Windows 11 will also feature new AI-driven experiences with Windows Copilot. Microsoft Launches Microsoft Fabric, the New Analytics Platform ‘for AI era’: Microsoft Fabric debuts as a comprehensive and integrated analytics platform designed to meet the diverse needs of organizations. This end-to-end solution seamlessly combines various data and analytics tools, including Azure Data Factory, Azure Synapse Analytics, and Power BI, into a single unified product. Fabric empowers data and business professionals to maximize the value of their data, enabling them to delve deeper into insights and enhance decision-making processes.  OpenAI Launches $1M Grants Program for Democratic Inputs to AI: OpenAI has announced that it will fund ten grants of $100,000 each, aimed at supporting experiments in establishing a democratic framework for determining the guidelines that govern the behavior of AI systems while staying within legal boundaries. Recognizing that AI’s impact will be “significant” and “far-reaching,” the ChatGPT creator wants decisions concerning how AI behaves to be influenced by diverse public perspectives. The deadline to submit the grant application is June 24, 2023. Microsoft Releases AI Governance Report: Microsoft has published a report titled "Governing AI: A Blueprint for the Future," which outlines guidelines for governments in formulating policies and regulations related to AI. The report emphasizes five key areas for consideration, including the creation of “fail-safe safety brakes” for AI systems that control critical infrastructure including city traffic systems and electrical grids to ensure AI is always under human control. The report highlights Microsoft's commitment to ethical AI practices and how the company is implementing responsible AI principles within its operations. Scientists Harness AI to Unleash Powerful Antibiotic Against Deadly Superbug: Scientists have utilized artificial intelligence (AI) to identify a new antibiotic capable of combating a dangerous superbug. In a study published in Nature Chemical Biology, researchers from McMaster University and MIT discovered a promising antibiotic, named abaucin, through the use of AI algorithms. The superbug in question, Acinetobacter baumannii, poses a severe threat to human health. The AI screening process enabled the identification of several potential antibiotics, with abaucin ultimately proving effective in suppressing the infection in laboratory tests.  NVIDIA Unveils DGX GH200 AI Supercomputer to Revolutionize Generative AI and Recommender Systems: NVIDIA has introduced the DGX GH200 AI Supercomputer, a groundbreaking innovation that combines 256 Grace Hopper Superchips into a single, massive GPU, capable of delivering 1 exaflop of performance and 144 terabytes of shared memory. With advanced NVLink interconnect technology and the NVIDIA NVLink Switch System, the DGX GH200 empowers researchers to develop next-generation models for generative AI language applications, recommender systems, and data analytics workloads. Expert Insights from Packt Community The AI Product Manager's Handbook – By Irene Bratsis Succeeding in AI – how well-managed AI companies do infrastructure right Many large technology companies that depend heavily on ML have dedicated teams and platforms that focus on building, training, deploying, and maintaining ML models. The following are a few examples of options you can take when building an ML/AI program: Databricks has MLflow: MLflow is an open source platform developed by Databricks to help manage the complete ML life cycle for enterprises. It allows you to run experiences and work with any library, framework, or language.  Google has TensorFlow Extended (TFX): This is Google’s newest product built on TensorFlow and it’s an end-to-end platform for deploying production-level ML pipelines. It allows you to collaborate within and between teams and offers robust capabilities for scalable, high-performance environments. Uber has Michelangelo: Uber is a great example of a company creating their own ML management tool in-house for collaboration and deployment. Earlier, they were using disparate languages, models, and algorithms and had teams that were siloed. After they implemented Michelangelo, they were able to bring in varying skill sets and capabilities under one system.  The above content is extracted from the recently published book titled "The AI Product Manager's Handbook," authored By Irene Bratsis and published in Feb 2023. To get a glimpse of the book's contents, make sure to read the free chapter provided here, or if you want to unlock the full Packt digital library free for 7 days, try signing up now! To learn more, click on the button below.     Sit Back, Relax and Read More Here! Secret Knowledge: AI/LLM Resources LLMs Enabling Self-Healing Software that Repair Vulnerabilities Automatically: Researchers have introduced a groundbreaking solution that utilizes Large Language Models (LLMs) and Formal Verification techniques to automatically detect and fix software vulnerabilities. The method involves Bounded Model Checking (BMC) to identify vulnerabilities and generate counterexamples that highlight incorrect system behavior. These counterexamples, along with the source code, are then fed into an LLM engine, which uses a specialized prompt language for code debugging and generation. The repaired code is verified using BMC.  Google Research Introduces MatCha to Revolutionize Computer Understanding of Visual Language and Chart Reasoning: MatCha is a groundbreaking pixels-to-text foundation model that aims to improve computer understanding of visual language, including charts and graphs. Training on chart de-rendering and math reasoning tasks, MatCha surpasses previous models in ChartQA performance by over 20% and achieves comparable results in summarization systems with significantly fewer parameters. The research papers on MatCha and DePlot will be presented at ACL2023, and the models and code are available on Google Research's GitHub repository.  Dialogue-guided intelligent document processing with foundation models on Amazon SageMaker JumpStart: A dialogue-guided approach to intelligent document processing (IDP) using Amazon SageMaker JumpStart. IDP automates the processing of unstructured data and offers improvements over manual methods. The solution discussed in the article combines OCR, large language models (LLMs), task automation, and external data sources to enhance IDP workflows. Incorporating dialogue capabilities and generative AI technologies, the system becomes more efficient, accurate, and user-friendly.  Resolving Code Review Comments with Machine Learning: Google has implemented a machine learning (ML) system to automate and streamline the code review process, reducing the time spent on code reviews. By training a model to predict code edits based on reviewer comments, Google's system suggests code changes to authors, increasing their productivity and allowing them to focus on more complex tasks. The model has been calibrated to achieve a target precision of 50% and has successfully addressed 52% of comments in offline evaluations.  MasterClass: AI/LLM Tutorials Build LLM-powered chatbot in 5 minutes using HugChat and Streamlit: If you’re interested in building a chatbot using Language Models, this is a step-by-step guide on developing an LLM-powered chatbot using HugChat, a Python library that simplifies the integration of LLMs into chatbot applications and Streamlit, a user-friendly framework for creating interactive web applications.  Unlock the Potential of Unstructured Data with BigQuery Object Tables: Discover how Google Cloud's BigQuery Object Tables, now generally available, empower AI developers to analyze unstructured data more effectively. Object tables provide a structured record interface for unstructured data stored in Cloud Storage, enabling the use of SQL and AI models for processing and managing diverse data types. You can access Google’s guided lab and tutorials to get started with your project. Vertex AI Embeddings for Text: Grounding LLMs Easily: Explore the concept of grounding and learn about Vertex AI Embeddings for Text and Matching Engine, including its key features. Learn how to build reliable Gen AI services for enterprise use, enabling deep semantic understanding and enhancing user experiences in applications such as search, classification, recommendation, and clustering. You can access the Vertex AI Embeddings for Text API documentation here and see the Stack Overflow semantic search demo on GitHub. Getting Started with Generative AI Studio on Google Cloud: Google Cloud offers Generative AI Studio, a user-friendly console tool for prototyping and testing generative AI models. This article provides step-by-step instructions on using Generative AI Studio through the Google Cloud user interface, without the need for REST API or Python SDK. Further resources are available in the GitHub repository for those interested in learning more about using Generative AI Studio.  HackHub: Trending AI Tools SamurAIGPT/privateGPT: Create a QnA chatbot on your documents without relying on the internet by utilizing the capabilities of local LLMs with complete privacy and security.  facebookresearch/fairseq: A sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling, and other text generation tasks.  iperov/DeepFaceLive: Swap your face from a webcam or the face in the video using trained face models. geohot/tinygrad: Aims to be the easiest deep learning framework to add new accelerators to, with support for both inference and training.  OpenGVLab/InternGPT: A pointing-language-driven visual interactive system, allowing you to interact with ChatGPT by clicking, dragging, and drawing using a pointing device.