Get in touch with our support team for any questions not answered in our help center.
Join our community on
Get in touch with our support team for any questions not answered in our help center.
Join our community on
Analyze streaming data with the pattern, category, and relationship power of graph analysis.
Developed by DARPA for advanced persistent threat (APT) detection, funded by Crowdstrike, thatDot Streaming Graph takes in multiple data streams at once, searches for important patterns formed from past data, data streaming past now, and data that hasn’t arrived, then pushes forward the key insights, the dangerous needles in the data haystack.
Instead of first converting categorical data into bloated numerical data, do deep analysis on IP addresses, people, things, etc. directly.
Use as both a source and a sink for Apache Kafka, Kinesis, SQS pipelines, and more. Pluggable, interchangeable storage options – Cassandra, Clickhouse, …
Read and write data fast, and handle out of order data with parallelized distributed compute.
Answer tough relationship, anomaly, and pattern questions with the standard Cypher graph query language and robust APIs.
Find patterns, anomalies, and problems without time windows. Query your entire historical data set plus current data in real-time.
Get unlimited scale – tested throughput of over 1 mill events/second. Supernode problem solved – tested to 100’s of millions of edges.
What Its For
As data flows in from multiple sources, use Streaming Graph to interpret that data as a graph, to find duplicates, relationships, and categories across the data streams. Set up a standing query that finds the pieces to the puzzle as they flow by. The moment the last piece clicks into place for an anomaly or pattern you've indicated, that data is pushed forward into a new data stream immediately. The mean time to answer (MTTA) can be measured in microseconds.
Time window software limitations have left our agencies, companies, and infrastructure utilities wide open to attack by bad actors, foreign and domestic, willing to find a way in and wait till those time windows expire. DARPA helped develop Quine, the open source heart of Streaming Graph, so you can spot insider threats, advanced persistent threats (APTs), etc. no matter how much time passes before exfiltration. Catch them in the act, the moment they start stealing data or damaging internal systems, not days or even months later after the damage is done.
Match up disparate device data streams with contextual data that may be stationary, resolve duplicates, intelligently filter out the unimportant, and analyze patterns of behavior for problems, bottlenecks, etc. Shift analysis left into the data stream itself to shorten Mean Time to Answer (MTTA).
As data from point of sale devices, ATMs, etc flows in, letting it get prepped and dropped in a stationary database before you can analyze it delays fraud analysis until it's too late to prevent, delays risk analysis until it may be too late to adjust your portfolio without loss. Spot fraudulent patterns or shifts in risk exposure within microseconds, while the data is still flowing in.
First
Ingest queries turn incoming data into a graph. First, they indicate what type of data is streaming in, Kafka, Kinesis, SQS, etc. or stationary data from files or more commonly, both at once. Define which topics, files, and devices you want, and finally, define which pieces of data become nodes, relationships, properties, etc. in the graph.
Second
Define the pattern to watch for with a standing query in the Cypher graph query language invented by Neo4J, plus some additions from thatDot to make it work for flowing data. Standing queries don’t depend on snapshots, time windows, or even that a pattern has occurred in the past. As the data flows through, the graph will morph and change, and the standing query will watch for the pattern you’re looking for to occur.
Third
As the data flows in, the graph grows more and more detailed and the standing query watches for the pattern you defined to appear. When the final piece of data completes that pattern, the results are sent within microseconds to where you indicated, usually through a new Kafka or Kinesis stream into an application that can act, or a monitoring system.
Fourth
Robust APIs and easy integration with existing data streams like Kafka mean the patterns found by Streaming Graph can be embedded in applications, workflows, or enterprise data architectures seamlessly. Trigger actions in real-time, alert subject matter experts, and push key information to monitoring software.
Finally
Use pluggable data persistence layers such as Clickhouse or Cassandra, or any Cassandra compatible database, to keep the new insights you’ve gained for later re-analysis, the data needed to regenerate the graph in case of power outages, for audit purposes, or to train machine learning algorithms such as graph neural network (GNN).
First
Ingest queries turn incoming data into a graph. First, they indicate what type of data is streaming in, Kafka, Kinesis, SQS, etc. or stationary data from files or more commonly, both at once. Define which topics, files, and devices you want, and finally, define which pieces of data become nodes, relationships, properties, etc. in the graph.
Second
Define the pattern to watch for with a standing query in the Cypher graph query language invented by Neo4J, plus some additions from thatDot to make it work for flowing data. Standing queries don’t depend on snapshots, time windows, or even that a pattern has occurred in the past. As the data flows through, the graph will morph and change, and the standing query will watch for the pattern you’re looking for to occur.
Third
As the data flows in, the graph grows more and more detailed and the standing query watches for the pattern you defined to appear. When the final piece of data completes that pattern, the results are sent within microseconds to where you indicated, usually through a new Kafka or Kinesis stream into an application that can act, or a monitoring system.
Fourth
Robust APIs and easy integration with existing data streams like Kafka mean the patterns found by Streaming Graph can be embedded in applications, workflows, or enterprise data architectures seamlessly. Trigger actions in real-time, alert subject matter experts, and push key information to monitoring software.
Finally
Use pluggable data persistence layers such as Clickhouse or Cassandra, or any Cassandra compatible database, to keep the new insights you’ve gained for later re-analysis, the data needed to regenerate the graph in case of power outages, for audit purposes, or to train machine learning algorithms such as graph neural network (GNN).