Redis Pipelines and Transactions with Golang

Redis is an in memory datastore mostly used as a cache. Clients can send commands to server using TCP protocol and get the response back. So, usually a request works like this.

• Client sends a request to server and waits for the response to be sent back.
• Server processes the command and writes the response to the socket for client to read.

Sometimes, in application flow we have to process multiple keys at once. In this case, for each request there will be a network overhead for the round trip between server and client. We can reduce this network overhead by sending commands to the Redis server in a batched manner and then process all the responses at once. This can be achieved using pipelines as well as transactions.

Pipelining in Redis is when you send all the commands to the server at once and receive all the replies in one go. It does not provide any guarantees like no other commands will be processed between your pipelined commands.

Transactions in Redis are meant to be used when you need guarantee that no other commands will be processed between your block of commands. A transaction is started by sending MULTI keyword to the server.

Let us see examples of how to execute both in Golang using Redigo as redis client.

• Connect to Redis server first. You can change the host parameter to the IP address/DNS of your Redis server.

import (
   "github.com/gomodule/redigo/redis"
   "time")

type RedisCache struct {
   pool *redis.Pool
}

var (
   // Pool :
   pool      *redis.Pool
   RedisCacheClient *RedisCache
)

func init() {
   pool = newPool(<host>, "")
   RedisCacheClient = &RedisCache{pool: pool}
}

func newPool(server string, password string) *redis.Pool {
   return &redis.Pool{
      MaxIdle:     3,
      MaxActive:   200,
      IdleTimeout: 240 * time.Second,
      Wait:        true,
      Dial: func() (redis.Conn, error) {
         c, err := redis.Dial("tcp", server, redis.DialPassword(password),
            redis.DialReadTimeout(5000*time.Millisecond), redis.DialWriteTimeout(5000*time.Millisecond), redis.DialDatabase(10))
         if err != nil {
            return nil, err
         }
         return c, nil
      },

      TestOnBorrow: func(c redis.Conn, t time.Time) error {
         _, err := c.Do("PING")
         return err
      },
   }
}

• Redigo support pipelining using the Send, Flush and Receive methods. Send writes the command to the connection’s output buffer. Flush flushes the connection’s output buffer to the server. Receive reads a single reply from the server. Here is a simple pipeline using these constructs:

conn := RedisCacheClient.pool.Get()
defer conn.Close()

//write all commands to connection's output buffer
conn.Send("SET", "foo1","bar1")
conn.Send("SET", "foo2", "bar2")
conn.Send("GET", "foo3")

//flush output buffer to server
conn.Flush()

//receive replies one by one 

reply1, err1 := conn.Receive()
reply2, err2 := conn.Receive()
reply3, err3 := conn.Receive()

• Transactions can be implemented using SEND and DO methods. Do will write the command it receives as argument to connection buffer and flush the entire buffer to server. All the replies will be received as response to DO command. For example:

conn := RedisCacheClient.pool.Get()
defer conn.Close()

//start transaction
conn.Send("MULTI")

//add commands to the output buffer
conn.Send("INCR" , "foo1")
conn.Send("INCR", "foo2")
conn.Send("GET", "foo1")

//Flush all the commands to output buffer and read replies
reply, err := conn.Do("EXEC")

Pipelines without transactions give you better performance then transactions due to the fact that other commands can be executed between the pipelined block where as transactions ensure that this will not happen. So, use transactions if you need atomicity guarantees else go with pipelines for enhanced performance.