Skip to main content

Fast Cache

note

Transform's Fast Cache is a release candidate feature and not generally available. Please reach out to support@transformdata.io to learn more.

What is the Fast Cache in Transform#

The Transform Fast Cache Feature builds on our materialization capabilities. The fast cache stores materializations in a DB-based cache (e.g., MySQL) for faster access. This is in contrast to the normal materializations, which by default store data in your data warehouse, and can have latency of many seconds. This feature is most useful for organizations that need low latency access to metrics. This is in contrast to the normal materializations, which by default store data in your data warehouse and can have a higher latency even for basic select-queries.

The Fast Cache Feature can be thought of as an additional capability ontop of the main Materialization Feature.

Fast Cache Components#

  1. Configuration & Definition:The Fast Cache definition occurs in the materializations object as a destination.
  2. Calling a Fast Cache: When the Fast Cache is specified as a destination for a materialization, it will be generated after the table is built using the mql materialize endpoint of the CLI and Python Interface.
  3. Writing the Fast Cache table: The Materialization endpoint will write the table to the Fast Cache.

Requirements#

  • The metrics you define in the Fast Cache configuration must already be defined in Transform.
  • There must be a primary time dimension that is consistently named across your data sources. This time dimension will be used to set the date parameters of the Fast Cache.
  • You must have provisioned a database for Transform to use as the Fast Cache. Please refer to our setup documents for more information.

Examples#

Defining a Fast Cache Destination#

Here is an example of a Fast Cache Configuration that reflects data from a booking service company.

materialization:  name: user_bookings_summary # name your materialization. this will be the table name written in your data warehouse.  description: # add an optional description.  destinations:    - location: fast_cache      format: wide  metrics: # list all the metrics you want to materialize in your materialization. all metrics must be defined in your metrics.yaml.    - bookings    - booking_value    - customer_service_tickets    - guest_host_messages
  dimensions: # list all corresponding dimensions you want to include for these metrics. these must all be defined in your data sources.    - ds    - guest__signup_channel

Command Options

Same as Materializations!

Backfills#

Backfills should be done manually by running materialize with the backfill dates. If you don't provide a date range, it will materialize the data for all time. This may be expensive and so the CLI will prompt you to confirm.