Priming

What is a Cache?

A cache is a temporary memory storage location which holds data that you want to be able to load and unload quickly. A helpful analogy is to compare a database cache to the RAM of a computer.

What is Priming a Cache?

Priming a Cache is when you load data from a database or other location into a cache. This allows the data to be returned from the cache layer, instead of going to the source database. It should reduce the time needed to generate a result.

When does Transform Prime?

For data sources that use a SQL Query, Transform's framework makes an optimization to prevent the query from executing multiple times if not necessary. A common example of the benefit of this optimization is if multiple metrics need data from a single data source, the query is not run multiple times to retrieve the data from those metrics. This process is beneficial especially when queries may be doing full table scans and are expensive to run. In certain cases, some configuration in the data source is needed to tell Transform's framework how to prime in order to get the best performance.

As noted in the best practices guide, it is always best to use SQL Table instead of SQL Query to define a data source. If a source is defined as a SQL Table, these priming queries will not be necessary, which will save computational cost.

:::