Building GenAI infra sounds cool—until it’s 3am and your LLM is down.

This free guide helps you avoid the pitfalls. Learn the hidden costs, real-world tradeoffs, and decision framework to confidently answer: build or buy? Includes battle-tested tips from Checkr, Convirza & more.

Here's what's happening in the world of AI, which has been buzzing with groundbreaking developments! This week, we're tracking OpenAI's global partnerships for democratic AI, the transparency debate sparked by Anthropic's Claude 3.7 prompt leak, and Google's powerful Gemini 2.5 Pro debut alongside a fresh 'G' logo. We also explore the intersection of tech and Saudi investment, a surprising Microsoft-Google collaboration for AI agent interoperability, Anthropic's real-time web search integration into Claude, and OpenAI's practical guide for enterprise AI adoption.

Ready to explore the cutting edge? Let's dive into the most captivating stories making headlines in the world of AI right now.

LLM Expert Insights,

Packt

With the dominance of LLMs, it may seem like we’ve acquired a magic wand capable of solving nearly any task — from checking the weather to writing code for the next enterprise solution.
In this context, one might wonder: are our favorite Python libraries, which we've long relied on, still relevant?

Today, we’ll talk about one such library, spaCy.

Despite the rise of LLMs, spaCy remains highly relevant in the NLP landscape. However, its role has

evolved. It now serves as a faster, more efficient, and lightweight alternative to large language models for many practical use cases.

Consider, for example, an HR screening system at a Fortune 500 company. spaCy can extract information such as names, skills, experience and other relevant details from resumes, and even flag profiles that best match a particular job description. Now imagine the cost per resume if, instead of spaCy, an LLM handled these tasks. spaCy excels at tokenization, part-of-speech

(POS) tagging, named entity recognition (NER), dependency parsing, and even building custom components using rule-based or machine learning-based annotators.

In this issue, we’ll briefly explore the spaCy NLP pipeline, as detailed in the Packt book, Mastering spaCy, Second Edition, by Déborah Mesquita and Duygu Altinok.

Here’s a high-level overview of the spaCy processing pipeline, which includes a tokenizer, tagger, parser, and entity recognizer.

1. Tokenization: Tokenization refers to splitting a sentence into its individual tokens. A token is the smallest meaningful unit of a piece of text — it could be a word, number, punctuation mark, currency symbol, or any other element that serves as a building block of a sentence.

Tokenization can be complex, as it requires handling special characters, punctuation, whitespace, numbers, and more. spaCy’s tokenizer uses language-specific rules to perform this task effectively. You can explore examples of language-specific data here.

Consider the following piece of code:

import spacy

nlp = spacy.load("en_core_web_md")

doc = nlp("I forwarded you an email.")

print([token.text for token in doc])

The tokens are:

['I', 'forwarded', 'you', 'an', 'email', '.']

2.POS tagging: Part-of-speech (POS) tags help us identify verbs, nouns, and other grammatical categories in a sentence. They also contribute to tasks such as word sense disambiguation (WSD). Each word is assigned a POS tag based on its context, the surrounding words, and their respective POS tags.

POS taggers are typically sequential statistical models, meaning the tag assigned to a word depends on its neighboring tokens, their tags, and the word itself.

To display the POS tags for the sentence in the previous example, you can iterate through each token as follows:

for token in doc:

print(token.text, "tag:", token.tag_)

The output for the example sentence is:

I tag: PRP

forwarded tag: VBD

you tag: PRP

an tag: DT

email tag: NN

3. Dependency parser: While POS tags provide insights into the grammatical roles of neighboring words, they do not reveal the relationships between words that are not directly adjacent in a sentence. Dependency parsing, on the other hand, analyzes the syntactic structure of a sentence by tagging the syntactic relations between tokens and linking those that are syntactically connected.

A dependency (or dependency relation) is a directed link between two tokens. Every word in a sentence plays a specific syntactic role, such as verb, subject, or object, which contributes to the overall sentence structure. This syntactic structure is heavily used in applications like chatbots, question answering, and machine translation.

In spaCy, each token is assigned a dependency label, just like other linguistic features such as the lemma or POS tag. A dependency label describes the type of syntactic relation between two tokens, where one token acts as the syntactic parent (called the head) and the other as its dependent (called the child).

For example, in the sentence “I forwarded you an email,” spaCy will label “I” as the subject performing the action, “you” as the indirect object (the recipient), “email” as the direct object, and “forwarded” as the main verb (or root) of the dependency graph.

A root word has no parent in the syntactic tree; it serves as the central verb that anchors the structure of the sentence.

Let’s look at how dependency relationships appear in this sentence:

for token in doc:

print(token.text, "\tdep:", token.dep_)

Output will be:

I dep: nsubj

forwarded dep: ROOT

you dep: dative

an dep: det

email dep: dobj

. dep: punct

If the sentence were “You forwarded me an email,” the direct and indirect objects would change, allowing us to capture the underlying relationships and perform further processing based on them.

Here are the dependency relationships for this sentence:

You dep: nsubj

forwarded dep: ROOT

me dep: dative

an dep: det

email dep: dobj

. dep: punct

4.Named Entity Recognition (NER): A named entity is any real-world object such as a person, a place (e.g., city, country, landmark, or famous building), an organization, a company, a product, a date, a time, a percentage, a monetary amount, a drug, or a disease name.

Some examples include Alicia Keys, Paris, France, Brandenburg Gate, WHO, Google, Porsche Cayenne, and so on. A named entity always refers to a specific object, and that object is distinguishable by its corresponding named entity tag.

For instance, in the sentence “Paris is the capital of France,” spaCy would tag "Paris" and "France" as named entities, but not "capital", because “capital” is a generic noun and does not refer to a specific, identifiable object.

Let’s see how spaCy recognizes the entities in the sentence in the following code snippet:

doc = nlp("I forwarded you an email from Microsoft.")

print(doc.ents)

token = doc[6]

print(token.ent_type_, spacy.explain(token.ent_type_))

Since Microsoft is the only named entity in the sentence, spaCy correctly identifies it and specifies its type.

[Microsoft]

This was just a quick peek into spaCy pipelines — but there’s much more to explore.

For instance, the spacy-transformers extension integrates pretrained transformer models directly into your spaCy pipelines, enabling state-of-the-art performance. Additionally, the spacy-llm plugin allows you to incorporate LLMs like GPT, Cohere, etc. for inference and prompt-based NLP tasks.

The book Mastering spaCy, Second Edition by Déborah Mesquita and Duygu Altinok is your comprehensive guide to building end-to-end NLP pipelines with spaCy.

Join Packt’s Accelerated Agentic AI Bootcamp this June and learn to design, build, and deploy autonomous agents using LangChain, AutoGen, and CrewAI. Hands-on training, expert guidance, and a portfolio-worthy project—delivered live, fast, and with purpose.

This is it.
50% off this Workshop ends on 18th May
If you’re in—move now.

Code: EXCLUSIVE50
Book Before 18th May Midnight

📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.

If you have any comments or feedback, just reply back to this email.

Thanks for reading and have a great day!

That’s a wrap for this week’s edition of AI_Distilled 🧠⚙️

We would love to know what you thought—your feedback helps us keep leveling up.

Thanks for reading,

The AI_Distilled Team
(Curated by humans. Powered by curiosity.)