Skip to main content

Druid Redis Cache

A cache implementation for Druid based on Redis.

Below are guidance and configuration options known to this module.

Installation

Use pull-deps tool shipped with Druid to install this extension on broker, historical and middle manager nodes.

java -classpath "druid_dir/lib/*" org.apache.druid.cli.Main tools pull-deps -c org.apache.druid.extensions.contrib:druid-redis-cache:{VERSION}

Enabling

To enable this extension after installation,

  1. include this druid-redis-cache extension
  2. to enable cache on broker nodes, follow broker caching docs to set related properties
  3. to enable cache on historical nodes, follow historical caching docs to set related properties
  4. to enable cache on middle manager nodes, follow peon caching docs to set related properties
  5. set druid.cache.type to redis
  6. add the following properties

Configuration

Cluster mode

To utilize a redis cluster, following properties must be set.

Note: some redis cloud service providers provide redis cluster service via a redis proxy, for these clusters, please follow the Standalone mode configuration below.

PropertiesDescriptionDefaultRequired
druid.cache.cluster.nodesRedis nodes in a cluster, represented in comma separated string. See example belowNoneyes
druid.cache.cluster.maxRedirectionMax retry count5no

Example

# a typical redis cluster with 6 nodes
druid.cache.cluster.nodes=127.0.0.1:7001,127.0.0.1:7002,127.0.0.1:7003,127.0.0.1:7004,127.0.0.1:7005,127.0.0.1:7006

Standalone mode

To use a standalone redis, following properties must be set.

PropertiesDescriptionDefaultRequired
druid.cache.hostRedis server hostNoneyes
druid.cache.portRedis server portNoneyes
druid.cache.databaseRedis database index0no

Note: if both druid.cache.cluster.nodes and druid.cache.host are provided, cluster mode is preferred.

Shared Properties

Except for the properties above, there are some extra properties which can be customized to meet different needs.

PropertiesDescriptionDefaultRequired
druid.cache.passwordPassword to access redis server/clusterNoneno
druid.cache.expirationExpiration for cache entriesP1Dno
druid.cache.timeoutTimeout for connecting to Redis and reading entries from RedisPT2Sno
druid.cache.maxTotalConnectionsMax total connections to Redis8no
druid.cache.maxIdleConnectionsMax idle connections to Redis8no
druid.cache.minIdleConnectionsMin idle connections to Redis0no

For druid.cache.expiration and druid.cache.timeout properties, values can be format of Period or a number in milliseconds.

# Period format(recomended)
# cache expires after 1 hour
druid.cache.expiration=PT1H

# or in number(milliseconds) format
# 1 hour = 3_600_000 milliseconds
druid.cache.expiration=3600000

Metrics

In addition to the normal cache metrics, the redis cache implementation also reports the following in both total and delta

MetricDescriptionNormal value
query/cache/redis/*/requestsCount of requests to redis cachewhatever request to redis will increase request count by 1