Complete Enterprise Product

Designed to make Data Engineering Teams exceed expectations and demands.

More Data Teams made 10x Productive

Enterprise Icon

Low-Code

Prophecy enables many more users - including visual ETL developers and Data Analysts. All you need to do is point-and-click and write a few SQL expressions to create your pipelines.

Enterprise Icon

COMPLETE

Low-Code Spark for workflow development. Low-Code Airflow for Scheduling. Metadata Search and Column Level Lineage for Management and Governance. We support complete Development & Deployment lifecycle

Enterprise Icon

CODE-BASED

As you use the Low-Code designer to build your workflows - you are developing high quality, readable code for Spark and Airflow that is committed to your Git.

Enterprise Icon

EXTENSIBLE

Prophecy gives you a gem builder - for you to quickly develop and rollout your own Frameworks. Examples are Data Quality, Encryption, new Sources and Targets that extend the built-in ones.

Prophecy Designer

Unique and intuitive design environment that helps you design superior workflows on Spark & schedules on Airflow

LOW-CODE DESIGNER

You can develop in visual view, and inspect and edit the code for Spark and Airflow.

Open Source

Generated Spark and Airflow code on Git is 100% open source format and runs without Prophecy.

Framework BUILDER

The built-in sources, targets, transforms and operators can be extended by providing templates as code with blanks to be filled in visual designer.

Interactive Debugging

Step-by-step interactive execution on Spark with sample data and live run of schedules on Airflow allows you to develop faster, running step-by-step to ensure it all works.

Tests

Unit tests and data quality test support is built in - making writing these tests super easy. The tests are run every time you commit code to maintain high quality.

Prophecy Hub

Enterprise-grade Metadata system that gets you faster and smarter insights about your data assets.

SEARCH & LINEAGE ACROSS ALL ASSETS

Your workflows and datasets store information about how the values are computed. Your schedules and executions store information about when they were computed. You can search all the information. You can track at column granularity with column-level lineage.

Extensible

Prophecy Hub is extensible. You can add information about the existing Entities such as Datasets (for example business definitions), or add other Entities (such as reports) so that all your information is in a single place and queryable.

Data Governance

Prophecy Metadata supports Data Governance by tracking PII and understanding computations of Columns.

Enterprise Icon

Column Level Lineage

View column level lineage at systems, project and workflow level. Per column, get the summary of transformations done on source data to produce its value.

Enterprise Icon

Tag Propagation

Propagate column level tags across data engineering pipelines to ensure that your sensitive data is appropriately tagged with privacy and security restrictions.

Enterprise Icon

Import External Sources

Add lineage for existing sources by running Prophecy Crawler on your Git repositories and your data stores. We  support Spark and Hive code.

Cloud Execution

Simplified Cloud Execution

Prophecy connects to your existing Spark cluster or more commonly connects to your account and spins clusters up and down as needed to run your workload.

ALL-Cloud Support

Prophecy runs on any cloud or on-premise. We support all major clouds. The same deployment can spin up Spark clusters in different clouds though your code is in one place.

EXECUTION Fabrics

Fabric is our virtual equivalent of on-premise big data clusters. This allows you to have separate environments for test, integration and production.

Secure Deployment

Prophecy installs on Kubernetes, whether on premise or on public cloud. In public cloud, we can run within your VPC, in addition to being available as a hosted service.

Agility with
Continuous Integration & Continuous Deployment

Bring the speed of DevOps to Data Engineering

Continuous Integration

Continuous Integration

Unit Tests

We make is simple to write unit tests - by generating a template to start per component. We can jump start the practice to get high coverage (90%+) rapidly and maintaining that. Every modification (git commit) and deployment runs unit tests to maintain high quality of workflows.

Continuous Deployment

Continuous Deployment

Data quality tests

We accelerate data quality tests, including auto-generation of suggested constraints to jump start data quality. If you are getting a detailed report of data quality all the time from test and production systems, you can deploy new changes with confidence.

Prophecy Transpiler

Convert legacy ETL workflows to Spark with high automation.

Prophecy Transpilers

Prophecy has transpilers, or cross compilers for formats and languages of Legacy ETL providers that provide highly automated modernization of your existing ETL workflows. Our expertise in moving multiple Enterprises allows you to move faster, more reliably and lower risk.