0

Explore Products

Best Sellers

New Releases

Books

Videos

Audiobooks

Free Learning

Blog .NET for Apache Spark UDFs Missing Shared State from Blog Posts - SQLServerCentral

Anonymous

60 min read
2020-10-29 18:10:20

0 Likes
0 Comments

article-image

The Problem When you use a UDF in .NET for Apache Spark, something like this code:

class Program { static void Main(string[] args) { var spark = SparkSession.Builder().GetOrCreate(); _logging.AppendLine("Starting Select"); var udf = Functions.Udf<int, string>(theUdf); spark.Range(100).Select(udf(Functions.Col("id"))).Show(); _logging.AppendLine("Ending Select"); Console.WriteLine(_logging.ToString()); } private static readonly StringBuilder _logging = new StringBuilder(); private static string theUdf(int val) { _logging.AppendLine($"udf passed: {val}"); return $"udf passed {val}"; } } Generally, knowing .NET we would expect the following output:

The post Blog .NET for Apache Spark UDFs Missing Shared State appeared first on SQLServerCentral.

Like
Save for later
Comment

0 Likes
0 Comments

Recommendations for you

Mathematics of Machine Learning

Mathematics of Machine Learning

May 2025 730 pages

eBook

£31.99 ~~£35.99~~

Building AI Agents with LLMs, RAG, and Knowledge Graphs

Building AI Agents with LLMs, RAG, and Knowledge Graphs

Jul 2025 560 pages

eBook

£31.99 ~~£35.99~~

Building Agentic AI Systems

Building Agentic AI Systems

Apr 2025 288 pages

eBook

£28.99 ~~£32.99~~

LLM Engineer's Handbook

LLM Engineer's Handbook

Oct 2024 522 pages

eBook

£36.99

Building Agents with OpenAI Agents SDK

Building Agents with OpenAI Agents SDK

Oct 2025 276 pages

eBook

£23.99 ~~£26.99~~

Node.js Design Patterns

Node.js Design Patterns

Sep 2025 732 pages

eBook

£31.99 ~~£35.99~~

Hands-On RTOS with Microcontrollers

Hands-On RTOS with Microcontrollers

Sep 2025 570 pages

eBook

£26.98 ~~£29.99~~

Generative AI with LangChain

Generative AI with LangChain

May 2025 480 pages

eBook

£31.99 ~~£35.99~~

AI Agents in Practice

AI Agents in Practice

Aug 2025 282 pages

eBook

£23.99 ~~£26.99~~

Learn Model Context Protocol with Python

Learn Model Context Protocol with Python

Oct 2025 304 pages

eBook

£23.99 ~~£26.99~~

article-image-from-c98-to-c23-the-arithmetic-mean-benchmarked-and-optimized

Ferenc Deak

27 Mar 2025

From C++98 to C++23: The Arithmetic Mean, Benchmarked and Optimized

Ferenc Deak

27 Mar 2025

10 min read

article-image-henrique-campos-on-game-development-patterns-with-godot-4

Henrique Campos

17 Jan 2025

Henrique Campos on Game Development Patterns with Godot 4

Henrique Campos

17 Jan 2025

5 min read

article-image-revolutionising-work-and-everyday-life-with-chatgpt

M.T. White

16 Dec 2024

Revolutionising Work and Everyday Life with ChatGPT

M.T. White

16 Dec 2024

10 min read

article-image-building-trust-in-ai-the-role-of-rag-in-data-security-and-transparency

Keith Bourne

13 Dec 2024

Building Trust in AI: The Role of RAG in Data Security and Transparency

Keith Bourne

13 Dec 2024

15 min read

Comments (0)

No comments for this article yet!