Handle Unusually Large Histories with Data Skew Protection
You do not always need to analyze every record or give AI every result to answer a question. zinzu provides controls that reduce how much work is done at different stages.
| Control | In plain language | What it controls |
|---|---|---|
| Data skew protection | Leave out unusually large histories that could dominate the work. | Which EventOwners reach sequence analysis. |
| Sampling | Analyze a smaller group of owners first. | How many EventOwners are included in the query. |
| AI sequence limit | Give AI a few examples instead of every matching sequence. | How much sequence data is included in an AI request. |
EventOwner means whoever or whatever you group records by: a customer, order, request, device, or application.
These controls can reduce processing work, but they are not a fixed spending cap. Cost also depends on the data read, query, compute resources, and AI request.
What Is Data Skew?
Imagine checking 1,000 customer folders. Most contain 20 pages, but one contains a million pages. That one folder can take far more work than the others.
The same thing can happen in your data. Most EventOwners may have a modest number of records, while a few have extremely large histories. This uneven distribution is called data skew.
For example, a busy device, automated account, or shared identifier might generate far more records than a typical owner.
How It Works
When enabled, zinzu counts each owner’s source records within the selected query time range, across the sources used by the query.
If an owner’s total exceeds the configured threshold, zinzu excludes that owner from sequence analysis for that run.
Example threshold:50,000 records per EventOwner
| EventOwner | Records in the selected range | What happens |
|---|---|---|
| Customer A | 120 | Eligible for analysis. |
| Customer B | 12,000 | Eligible for analysis. |
| Customer C | 50,000 | Eligible; exactly the threshold is allowed. |
| Customer D | 80,000 | Excluded from sequence analysis. |
Customer D’s entire history is excluded for that run. zinzu does not simply retain its first 50,000 records.
If one owner has 30,000 records in one source and 25,000 in another, the combined 55,000 exceeds this example threshold.
Why It Helps
Excluding exceptionally large histories can reduce the work and memory needed for sequence matching and prevent a small number of owners from dominating processing.
Counting those records still requires work. This protection does not mean the excluded data costs nothing to read.
When to Use It
Use it when you want to explore typical behavior and a few unusually large histories are making analysis expensive.
Review exclusions carefully when investigating heavy users, automated activity, fraud, or busy devices—the excluded owners may be the ones you need.
Important Notes
- The threshold applies to source records, not just records that match your event definitions.
- It applies within the selected query range, not the owner’s entire lifetime.
- No records are deleted from your source tables.
- Excluded owners do not contribute matches to that query.
- 50,000 is an example threshold, not a universal default.
Current availability:Data skew protection is implemented in the backend through query configuration. The reviewed query-builder UI does not expose a setting for it. It is disabled when no configuration enables it.