Microsoft releases version 1.0 of .NET for Apache Spark. The open source package is intended to connect .NET development with the platform for big data processing.

Apache Spark scales complex data analyzes and machine learning applications over a cluster. The software has established itself as a standard tool for the analysis of large amounts of data and is written in the Scala programming language.

Support for .NET applications In addition to support for .NET applications for .NET Standard 2.0 or older, the current release also offers the option of using user-defined functions from .NET (UDFs for short).

In addition, users get access to the Apache Spark DataFrame APIs (version 2.3, 2.4 and 3.0) and the ability to use Spark SQL – a processing module from Apache Spark and queries of structured data.

The big data engine is currently in its third major version. Further information on .NET for Apache Spark can be found in the release notes and in the article