Average Ratings 0 Ratings
Average Ratings 0 Ratings
Description
Deequ is an innovative library that extends Apache Spark to create "unit tests for data," aiming to assess the quality of extensive datasets. We welcome any feedback and contributions from users. The library requires Java 8 for operation. It is important to note that Deequ version 2.x is compatible exclusively with Spark 3.1, and the two are interdependent. For those using earlier versions of Spark, the Deequ 1.x version should be utilized, which is maintained in the legacy-spark-3.0 branch. Additionally, we offer legacy releases that work with Apache Spark versions ranging from 2.2.x to 3.0.x. The Spark releases 2.2.x and 2.3.x are built on Scala 2.11, while the 2.4.x, 3.0.x, and 3.1.x releases require Scala 2.12. The primary goal of Deequ is to perform "unit-testing" on data to identify potential issues early on, ensuring that errors are caught before the data reaches consuming systems or machine learning models. In the sections that follow, we will provide a simple example to demonstrate the fundamental functionalities of our library, highlighting its ease of use and effectiveness in maintaining data integrity.
Description
Scala seamlessly integrates both object-oriented and functional programming paradigms into a single, elegant high-level language. With its static type system, Scala minimizes the likelihood of errors in intricate applications, while its compatibility with JVM and JavaScript allows developers to create efficient systems that can leverage extensive libraries. The Scala compiler is adept in managing static types, meaning that in most instances, you don't need to specify variable types; its robust type inference handles this automatically. Structural data types in Scala are represented by case classes, which automatically provide well-defined methods for toString, equals, and hashCode, in addition to enabling deconstruction through pattern matching. Moreover, in Scala, functions are treated as first-class citizens, allowing for the creation of anonymous functions using a streamlined syntax. This versatility makes Scala an appealing choice for developers seeking a language that combines the best of both programming worlds.
API Access
Has API
No
API Access
Has API
Yes
Integrations
Apache Spark
Yes
Atom
No
Claude Opus 4.8
No
CodeFactor
No
CodeScene
No
Dash
No
ERNIE 4.5
No
ERNIE X1
No
GPT-5.6 Sol Ultrafast
No
GPT‑5-Codex
No
Integrations
Apache Spark
No
Atom
Yes
Claude Opus 4.8
Yes
CodeFactor
Yes
CodeScene
Yes
Dash
Yes
ERNIE 4.5
Yes
ERNIE X1
Yes
GPT-5.6 Sol Ultrafast
Yes
GPT‑5-Codex
Yes
Pricing Details
No price information available.
Free Trial
No
Free Version
No
Pricing Details
Free
Free Trial
No
Free Version
Yes
Deployment
Web-Based
Yes
On-Premises
No
iPhone App
No
iPad App
No
Android App
No
Windows
No
Mac
No
Linux
No
Chromebook
No
Deployment
Web-Based
Yes
On-Premises
Yes
iPhone App
No
iPad App
No
Android App
No
Windows
Yes
Mac
Yes
Linux
Yes
Chromebook
Yes
Customer Support
Business Hours
No
Live Rep (24/7)
No
Online Support
Yes
Customer Support
Business Hours
No
Live Rep (24/7)
No
Online Support
Yes
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Vendor Details
Company Name
Deequ
Website
github.com/awslabs/deequ
Vendor Details
Company Name
Scala
Website
www.scala-lang.org