Analytical Transaction Processing (ATP): Real-Time, Graph Decision-Making Made Easy

There are two “must-have” ingredients to making effective business decisions: a complete contextual view of the situation and real-time data. Carl Olofson, Research Vice President at IDC, writes, “We can build a very complete view of the enterprise and its business context, and act on that knowledge very quickly, even automatically. With a full contextual view of the business situation, together with live transactional data, we are in a position to make decisions ‘at the moment.’ Doing this, however, requires the coordination of operations and the leveraging of all relevant data in a single processing context…this capability is something we call ‘analytical transaction processing’ (ATP).”  In this article, we will examine how a native parallel graph (NPG) database is the best, and often the only way, to enable ATP. In fact, the ATP capability enabled by the graph is one of the key fundamental reasons why large enterprises are using NPG databases in production. Let’s start with a few examples of the limitations of current, non-ATP technology.  

Example 1: Credit Card/Loan Application Use Case

When an end-user fills in her application online, the system needs to record her application as a transaction to a database. In the highly innovative and competitive financial services industry, it’s vital to perform necessary risk assessments before returning a decision on an application — all as quickly as possible. Although legacy approaches, such as relational database management systems (RDBMSs) and key-value databases, have no problem recording and retrieving such transactions with high speed and throughput, there is something missing.  We need to “connect the dots” when performing risk assessments when the application is submitted. Connecting the dots means finding how the incoming application is connected to previously known fraudulent identities or applications. It also means detecting potential “bursty” applications from professional groups of fraudsters who are not strongly connected to previously known fraudulent identities but who are, as a group, exhibiting suspicious patterns. The system needs to be able to use the incoming and most recent applications in its ATP, which cannot be done by offline systems (such as Hadoop or other systems). How the “dots are connected” is unknown beforehand and the paths to connect dots can be complex. Connecting the dots is challenging for RDBMSs and key-value databases, but easily done using an NPG database. For example, a point-of-sale consumer finance platform for ecommerce is improving its ATP capabilities with an NPG database. Additionally, many of the largest banks are already taking advantage of ATP powered by NPG databases.  

Example 2: Power Grid Use Case

One recent breakthrough in the energy management industry is what experts call the “faster than real-time” power flow analytics. Any event within an electrical power grid (such as a pole down, a factory’s heavy machinery is turned on/off),  has a network effect on other components in the network (directly and indirectly), propagating at the speed of electricity. In electrical energy management production systems, important equipment has sensors to send back real-time measurements ( such as voltages and power) to the control centers, typically every 500ms or so. Each sensor represents a variable, and balancing/rebalancing the power grid requires fitting thousands of variables so that they simultaneously satisfy thousands of equations. Using an NPG database, the control center is able to run iterative computation to compute and predict the status of the network. If needed, the control center can intervene in less than 100ms. That’s why they call it faster than real-time: The time needed to compute and predict the status of every piece of equipment in the network is faster than the input they can collect! This is ATP in high gear. Not only must transactions (the sensor inputs) be stored in real-time but they also require fast computation (PageRank type of network analytics), which requires every data point in the database (information about every equipment such as power line, generator and user load) in many iterations. No RDBMS or key-value database can provide this type of ATP.  

Example 3: Healthcare Use Case

When a patient calls into a service center run by a major healthcare insurance provider, the company wants to gather everything about the patient as efficiently as possible.  This expedites patient care and reduces healthcare costs for everyone. Even more interestingly, the healthcare company may call the patient if the patient has missed appointments or prescription refills; this is a red flag that the patient is not adhering to the doctor-recommended path to wellness. Furthermore, the representative would like to find similar members with similar wellness journeys to have a better understanding of the case at hand. This wellness journey info also helps the healthcare rep be more proactive in providing affordable and quality healthcare services to the organization’s other members. Without a graph connecting all these data points, traditional RDBMS would have to join many tables — a process that is typically slow and often impossible. RDMS simply cannot analyze and match the patient journey in real-time across millions of patients and billions of claims.  

Example 4: Telephone Fraud Use Case

If a fraudster attempts to phone someone, the call should be flagged to the callee. Tagging phone numbers as bad or good doesn’t work to detect malicious phone calls, as professional criminal groups often use new phone numbers. False positives will, essentially, make the detection system unusable. With an NPG database, every incoming phone call is connected in the call database. This means many new features based on graph structures (such as how many hops separate the caller number from the callee number) are computed in real time by the graph DB and used to predict the likeness of fraud. A large mobile phone network is using an NPG database in this way. Again, an offline system or a RDBMS cannot handle this type of highly complex business logics in real-time.  Enterprises that embrace ATP powered by NPG graph technology will have unfair competitive advantages, as they are literally one step ahead when it comes to processing financial applications, providing quality healthcare, and preventing fraud.   Leading companies in banking, IOT and eCommerce are already seeing the bottom-line business benefits of ATP powered by NPG. Soon, the ATP-NPG combination will be a “must” for forward-looking enterprises’ technology portfolios– it’s not “if,” but “when.”

Author:

Dr. Yu Xu, CEO and Founder, TigerGraph