Enterprise  GENERATIVE AI

Well Architected   Framework & Patterns

An Architect’s Real-life Guide to Adopting Generative AI in Enterprises at Scale

SUVORAJ BISWAS

COPYRIGHT NOTICE

Copyright © 2023 by Suvoraj Biswas

            All rights reserved. No part of this publication may be reproduced, distributed, or transmitted in any form or by any means, including photocopying, recording, or other electronic or mechanical methods, without the prior written permission of the author, except in the case of brief quotations embodied in critical reviews and certain other noncommercial uses permitted by copyright law.For permission requests, reach the Author at suvoraj.biswas@gmail.com

            This book, " Enterprise Generative AI Architecture " is intended for informational purposes only. The content within is provided "as is" and without warranties of any kind, express or implied, including but not limited to accuracy, completeness, or fitness for a particular purpose. The author and publisher shall not be liable for any errors, omissions, or any losses, injuries, or damages arising from the display or use of this information.

The book's content, including all illustrated pictures, reflects the author's personal perspectives and research. It is entirely independent and not affiliated with any of the author's current or past organizations.

PREFACE

            Welcome to the " Enterprise Generative AI Well Architected Framework & Patterns " a comprehensive step by step guide designed for Enterprise IT Professionals to explore the cutting-edge world of generative artificial intelligence (AI) systems within the context of enterprise applications. As most of the businesses continue to embrace AI-driven solutions to streamline processes, drive automation and generate valuable insights, the importance of robust and scalable generative AI architectures becomes increasingly evident. When I decided to explore Generative AI, as a Solutions Architect I faced a lot of challenges, roadblocks and at the same time learned a lot while exploring many tools and technologies during the initial steps. My everyday learning and challenges pushed me to come up with generic well architected frameworks for my fellow IT professionals to build world class software to delight the Enterprise stakeholders following the existing common LLM integration and adoption pattern.

            This book is intended to provide readers with a clear understanding of the fundamental principles, methodologies, and best practices for implementing generative AI in large-scale enterprise environments. Whether you are a seasoned AI practitioner(Architects, Engineers / Engineering Managers or Product Managers) seeking to deepen your knowledge or an enterprise leader (VPs, CXOs, Founders) exploring the potential of Generative AI for your organization, this book offers valuable insights into leveraging the power of generative models effectively and responsibly. We will also discuss the challenges and ethical considerations associated with deploying AI systems in enterprise settings and provide strategies to address these concerns.

Please note that I have shared information about both open-source tools and products from premium software providers. This information is based on my independent research and exploration, and I am not affiliated with any entity or sponsored by anyone. Therefore, I recommend conducting your own due diligence before deciding to use any of these tools or products.

            I hope that " Enterprise Generative AI Well Architected Framework & Pattern " will serve as a valuable resource on your journey to unlocking the potential of generative AI in your organization. Feel free to connect with me at Linkedin ( www.linkedin.com/in/suvoraj ) or comment at Amazon to share your valuable feedback.

Happy reading & exploring Generative AI.

Suvoraj Biswas

ACKNOWLEDGEMENT

I want to express my deep appreciation to all the readers and reviewers of this book. Your belief in me and my work, along with your continued support throughout this journey, means the world to me.

I extend my heartfelt thanks to all my professors & supervisors in my career, who have been instrumental in my professional growth, and to my colleagues, from whom I've gained valuable technical and professional insights and knowledge.

Last but certainly not least, I am sincerely grateful to my wife, Moumita, and my two wonderful children, Shantosmita and Soujash, for their unwavering support. They sacrificed precious family time to support my late nights and early mornings, allowing me the space to think and write this book.

TABLE OF CONTENTS

Chapter 1:

Introduction to  Generative AI - The Well Architected Framework

• What is the background of GenAI - Well Architected Framework?
• Why do we need a Well Architected Framework for GenAI based Enterprise Use Case?
• What are the pillars of the Well Architected Framework?
• What are the building blocks for each pillar?

Chapter 2:

Operational Excellence Pillar

• What is a Large Language Model?
• What is a Foundation Model?
• When should we be using the Foundation Model?

Chapter 3:

Security & Privacy Pillar

• Introduction to Content Moderation building block

• What is AI Guardrails and How to integrate it?

Chapter 4:

Compliance Pillar

• Introduction to the Compliance Pillar:
• Benefits of having an Archival for Compliance building block
• What are some solutions for Archiving AI & Human interactions for Regulatory Compliance?

Chapter 5:

Reliability Pillar

• The FM or LLM observability Building Block:
• What are some LLM Observability Solutions?

Chapter 6:

Cost Optimization

• Introduction
• What are some best practices to optimize cost during Architecting?

Chapter 7:

System Architecture Excellence

• Prompt Engineering
• Why Prompt Engineering plays a crucial role in the GenAI apps?
• Some Best Practices for Effective Prompts:
• Embedding & Vector Database
• What are Embeddings and why are they so important?
• What are OpenAI's Offerings on Embeddings?
• What is a Vector Database & How do they really work?
• Options of Vector Databases for Solutioning
• The Orchestrator Building Block
• Some Orchestrator Application framework for AI development

Chapter 8:

Common Architectural Pattern For FM/LLM Integration & Adoption

• Introduction
• What is Retrieval-augmented generation (RAG) Pattern for GenAI?
• Demo Time - a GenAI based QnA python app using RAG pattern (LangChain, PineCone & Open AI api)
• What is Fine tuning ?
• Why do we need fine tuning? Can’t we use RAG for domain specific use cases?
• Are Fine Tuning & Pre-Training referring to the similar process?
• Foundation Model or Large Language Model fine-tuning techniques

• Unsupervised / Supervised Fine Tuning (U-SFT)
• RLHF Reinforcement Learning from Human Feedback
• Parameter Efficient Fine Tuning (PEFT)
• What is the difference between regular fine tuning (SFT or RLHF) and PEFT?
• What are some Enterprise Use cases where SFT/RLHF or PEFT can be used?

• What is LoRA (Low Rank Adaptation Model) technique?
• What are the advantages of the LoRA method in fine tuning LLM?
• What is QLoRA (Quantized Low Rank Adaptation Model)?

Chapter 9:

AWS Solutions for Generative AI

• Vector Database & Semantic Search capabilities
• How to run PostGreSQL with pgVector extension in EC2 ?
• Overview of AWS Sagemaker and AWS Bedrock
• Example Generative AI based Enterprise Product Search Architecture in AWS

Conclusions

References (Tools, Libraries, Articles)

WHAT INDUSTRY LEADERS & EXPERTS ARE SAYING ABOUT THIS BOOK?

“ Enterprise GENERATIVE AI Well Architected Framework & Patterns ” is a new addition in this very timely topic on Artificial Intelligence by Suvoraj Biswas. There are some other  publications in this area. What has impressed me about this edition is its readability and expression in a lucid manner. This book has explained to readers with a clear understanding of the fundamental principles, methodologies, and best practices for implementing generative AI in large-scale enterprise environments. Additional advantage provided in this book is the information about both open-source tools and products from premium software providers, gained by the author from many years of experience in the IT industry.

Dr Debajyoti Mukhopadhyay

Former Scientist at Bell Communications Research, New Jersey, USA

Former Dean of the School of Engineering & Applied Sciences, Bennett University, India

https://in.linkedin.com/in/debajyotimukhopadhyay

" Enterprise Generative AI Well Architected Framework & Patterns " is a comprehensive book that explores the intricacies of Building Gen AI Solutions with remarkable clarity and depth.The book covers the six building blocks that the architects should consider while building Gen AI solutions for enterprise.

In addition to the theoretical underpinnings, the book offers practical guidance for implementing Generative AI in real-world scenarios. The step-by-step examples and code snippets are exceptionally well-written, allowing readers to experiment and gain hands-on experience. It strikes the perfect balance between theory and practicality, making it an indispensable resource for anyone interested in the fascinating world of Generative AI.

Mitali Biswas

Chief Information Officer, CK Birla Hospitals, India

https://www.linkedin.com/in/connect-mitali-biswas

As LLM practitioners, most of our work focuses on how to leverage the existing models to output the results we want (the operational excellence) and overlook the other important aspects such as security, compliance, reliability, cost optimization, etc. With clear illustrations and real-world scenarios, this book not only broadens the horizon of LLM practitioners but also provides a cohesive way to tie different components into a sustainable and trustworthy LLM system. A must-read for anyone wanting to truly understand and harness the power of LLM in a comprehensive manner.

Leyuan Yu

Senior Machine Learning Scientist, Coursera, Canada

https://www.linkedin.com/in/leyuanyu/

The book is a comprehensive guide to explore how generative AI could be leveraged at the Enterprise level. It also simplifies the concept of embeddings, vector databases, and prompt engineering which are critical for successful adoption of generative AI. Certainly a good read for anyone looking to get started in this space.

Rudra Roy Choudhury

Product Manager @ Tubi, USA | Linkedin Top Voice

https://www.linkedin.com/in/rudravaswata-roychoudhury/

CHAPTER  

ONE

Enterprise Generative AI -  

The Well Architected Framework

What is Generative AI?

We have all witnessed how OpenAI has recently reshaped the digital landscape through the introduction of tools like ChatGPT, which gained a substantial user base surpassing all popular social media applications.  ChatGPT has been powered by what we call Generative AI which not only has remarkable influence in the consumer sphere but also many Enterprises are adopting to solve many business challenges which previously appeared impossible. Generative AI is a form of deep learning system which is able to generate new original contents (texts or digital media - audio or Video or images). It uses the machine learning algorithm and artificial neural networks to recognize the underlying pattern in the training data to predict new original contents without any human intervention or influences.  Before understanding Generative AI we need to understand the ecosystem first.

Artificial Intelligence, often abbreviated as AI, refers to a computer system capable of emulating human behavior and performing tasks without requiring explicit programming.

In the same ecosystem, we find Machine Learning (ML), which is a subset of AI. However, it has the ability to autonomously learn from historical data and make predictions based on the patterns it acquires. For example, through the utilization of Machine Learning, we can predict patterns such as whether people prefer takeout or dining in a restaurant during a particular season, based on historical data.

Similarly Deep Learning technique refers to another subset of Machine Learning which gets the cognitive ability in the computer systems by integrating the artificial deep neural networks just like the human brain. It entails a substantial collection of input data and hidden layers designed in a manner where the output from one layer is progressively refined in the subsequent layer, ultimately generating predictive contents.

Generative AI is the part of Deep Learning models which can generate completely new content in the form of texts or images or videos based on these supervised training data.

What is the background of GenAI - Well Architected Framework?

With the rise of ChatGPT and other Large Language Model based AI systems modern businesses are rapidly embracing the integration of Generative AI into their enterprise systems to solve many potential business challenges which previously were not possible.

As a solution architect I got the opportunity to work on several projects using Generative AI when my company decided to explore the Generative AI and Large Language Model for some potential Enterprise use cases. It has been a great learning experience for me, indeed. While exploring and designing the effective solutions we faced many kinds of challenges. Based on my learning and evaluating various aspects of Generative AI, I felt the lack of proper Architectural Framework and decentralized patterns for adopting generative AI. This brings the need to follow an uniform approach and patterns combined together so that Enterprises can focus on building robust cost effective AI products within their allocated budget and strict timeline.

Why do we need a Well Architected  Framework for GenAI based Enterprise Use Case?

A Well Architected Enterprise Generative AI Framework, embedded within the enterprise architecture, offers a structured and coherent methodology, which if adopted correctly can help mitigate the business and technical challenges while adopting the Generative AI capabilities. The framework being proposed here is constructed upon a standardized collection of building blocks and supporting pillars that facilitate the conception, implementation and management of the enterprise-grade Generative AI-based solutions. Here's why such a framework is crucial to adopt:

• Consistency and Efficiency  - by adopting this framework your organization would be able to follow the uniform approach and pattern to build consistent and efficient products for all kinds of Gen AI needs.

• Scalability - As the end user usage increases for the delivered products, the complexity of both the operation and maintainability increases. Adopting a predefined framework during the build stage addresses the long term scalability requirement.

• Integration - For a large enterprise a specific solution might need to integrate with many different input data sources or systems. Enterprise Generative AI framework addresses this requirement at an early stage by allowing a loosely coupled approach and  smooth data flow.

• Agility - The framework addresses the rapidly evolving customer requirement, usage pattern and allows the room to respond and adjust swift without compromising the stability of the delivered Gen AI  product.