Why Redis Pipelining is a Good Idea?

What is PipeLining?

Pipelining is a form of asynchronous task execution. A Pipeline is a task that is composed of many subtasks. Each subtask may be dependent on its previous subtasks.

t1 => t2 => t3
            ^
        t4 =|

t2 depends on t1. t3 is dependent on t2 & t4.

While we are executing subtask t3 for primary task S1, we can also execute subtask t1 for another primary task S2. This is a pipelined execution.

How Redis Pipelining helps in Client-Server communication?
Each network request between the client and server involves a latency named Round-Trip-Time. Redis server reads a request from a client and client waits till the server writes the response.
We can rather ask the client to send many requests at once and collectively wait for the responses.

This helps us achieve:

  • Ability to counter RTT of a network.
  • Better throughput for client
  • Saves multiple reads from server to client.
  • The server sends single write with all responses

Classic Usage

  • POP3 uses pipelining

References

Advertisements

Notes on Redis DB

  • Redis provides atomic operations on keys
  • Keys can have TTL (-2 expired, -1 never expire, n seconds to expire)
  • Values are stored as a list.
    • The list is indexed and can be queried with ranges.
  • List items can be popped, Left/Right pushed.
  • Values can also be stored as set.
    • Set operations: add, remove, is_member, members, union
  • Values also support sorted sets, using a user defined sorting key
  • Another value type is Hashed values.
    • key : {key: value, key:value}
    • HSET user name "test" address "earth"
    • HGET user name
    • Atomic operations are available on hash values too.
  • Keys can optionally follow a schema.
    • usr:123:id
    • An O(N) search is possible on a prefix of the key

Reference

The Best Go Library for Redis

There are two popular choices:

  • go-redis
  • redigo

I prefer redigo over go-redis.

Cons with go-redis

  • Not enough documentation for APIs
  • Difficult to understand naming conventions for APIs
  • I’m still finding how to refresh expiry time of a key using go-redis
  • Needs Cmdable interface for calling Redis commands. The interface is not so well designed.

An interface in go-redis

   type Cmdable interface {
   TTL(key string) *DurationCmd
   Restore(key string, ttl time.Duration, value string) *StatusCmd

Pros with redigo

  • Easy to understand APIs
  • Ability to call Redis commands from the client so you never have to learn a new wrapper
  • No extra interface like go-redis

A sample client with go-redis is as following:

  package main

  import (
      "fmt"

      "github.com/go-redis/redis"
  )

  func HandleErr(err error) {
      if err != nil {
          panic(err)
      }
  }

  /*
  type DurationCmd struct {
      baseCmd

      val       time.Duration
      precision time.Duration
  }
  */

  func ExampleNewClient() {
      client := redis.NewClient(&redis.Options{
          Addr:     "localhost:6379",
          Password: "", // no password set
          DB:       0,  // use default DB
      })
      
      err = client.Set("key", "value", 0).Err()
      HandleErr(err)

      val, err := client.Get("key").Result()
      HandleErr(err)
      fmt.Println("key", val)

      ttl := client.TTL("key")
      fmt.Println("ttl", ttl)
  }

  func main() {
      ExampleNewClient()
  }

Install Redis on Mac OS via Homebrew

Install Redis on Mac OS via Homebrew

  • Install redis package
    $ brew install redis
    Updating Homebrew...
    ==> Auto-updated Homebrew!
    
  • After installation, you’d see the following message:
     ==> Downloading https://homebrew.bintray.com/bottles/redis-5.0.4.mojave.bottle.tar.gz
     ############################################################# 100.0%
    ==> Pouring redis-5.0.4.mojave.bottle.tar.gz
    ==> Caveats
    To have launchd start redis now and restart at login:
      brew services start redis
    Or, if you don't want/need a background service you can just run:
    redis-server /usr/local/etc/redis.conf
    
  • Run redis as following:
    $ redis-server /usr/local/etc/redis.conf
    oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
    

Connect to Redis with redis-cli

redis-cli 5.0.4

Usage: redis-cli [OPTIONS] [cmd [arg [arg ...]]]
  -h <hostname>      Server hostname (default: 127.0.0.1).
  -p <port>          Server port (default: 6379).
    -n <db>            Database number.
$ redis-cli -h <host> -p <port> -n <db>

Time based Key Expiry in Redis

https://redis.io/commands/expire
It is a useful feature to expire keys based on their last access time. We can use it to develop interesting feature such as rate limits,

There are various rate limiting implementations.
https://github.com/redislabsdemo/RateLimiter/tree/master/src/com/redislabs/metering/ratelimiter

Written with StackEdit.