Streaming Graph ETL

The Problem
Most ETL tools use the batch processing paradigm to find high-value patterns in large volumes of data. Whether the specific business application is fraud detection, cyber security, network observability, e-commerce or ad targeting, batch processing translates into delay. Even if you are processing data in small batches, you are missing opportunities to react to events as they happen and shape outcomes in ways beneficial to your business. A great example is insider trading. The cost of detecting someone who is about to execute an insider trade is much less than the cost of trying to unwind that trade later when batch processing picks it up. Even if the batch process runs every five minutes, that just means you'll find them sooner, not stop them. Ultimately stream vs. batch will result in the costly reversal of transactions, not stopping them in real-time.
The Solution
Streaming ETL using Quine means not just knowing but acting on events as they occur. Use Quine's ingest queries to materialize event data as a graph, with a graph’s ability to express and query for complex relationships between seemingly unrelated data. Then use Quine’s standing queries to monitor for key patterns (e.g. indicating a fraudulent transaction or cyber attack is underway) and take action when those patterns emerge. Quine’s graph ETL also makes it straightforward to process categorical data — everything from email addresses and model numbers to IP addresses and process IDs — that other systems ignore or try to encode. Use Quine Enterprise to scale your graph ETL to millions of events per second.

Key Value Delivered

Use standing queries to detect patterns as they occur and take action
Join data from multiple sources as scale
Resolve entities across sources
Mitigate out-of-order data arrival
De-duplicate data
Generate new events from data as it streams, in real-time
Integrates with existing Apache Kafka, AWS Kinesis, data lake, and API event sources.

What people are saying

thatDot Platform’s advanced Novelty Scoring for categorical data feature is the future of anomaly detection. Its speed powers our new real-time services while also significantly reducing false-positive findings for our customers.

Gery Szlobodnyik

CEO

The security industry is headed towards leveraging knowledge graphs of all the relevant evidence provided by threat groups, network indicators and endpoint artifacts to quickly identify and mitigate threats. The security companies that can most confidently extract relevant context from these graph and then immediately action on findings to mitigate threats will be the winners. This is exactly the problem Quine aims to address.

Evan Wright

Staff Data Scientist

See for yourself

If you think Streaming Graph or Novelty might be for you, contact us to see them in action.

Streaming Graph ETL

The Problem

The Solution

Key Value Delivered

Recent posts

Microservice Hell: The State of the Art in Streaming Services

4 Advantages to Streaming Analytics in Graph Form

Start building

Read the docs

View the demos

Join our community

What people are saying

See for yourself