Notes on String & Encoding Techniques

String and their encoding decide the languages the code can support.

Introduction

  • We have many languages and their symbols that need more than 8-bits (ASCII) for binary representation.
  • Encoding adds semantics to a set of bytes.
  • Unicode is a table of all characters and their numeric equivalent.
  • Since there are more than 100k symbols, 8-bits are not enough.

What is UTF-8

  • Unicode representation of a string needs 12-bits, padded to 32-bit.
  • The idea of UTF-8 is a variable encoding of symbols. The common ASCII symbols can use just 8-bits and other extended symbols can use up 1,2,3 or 4-bytes.
  • UTF-8 is more memory efficient compared to UTF-16 which uses minimum two bytes for a symbol.

How do Programs Use an Encoding?

  • The program picks a library for string encoding.
  • UTF is the most common library used.
  • The number of bytes in encoded string is decided by number of leading 1s in the array.

Test

见/見
Unicode= \U+FFE8\U+FFA7/\U+FFE8\U+FFA6

References

Written with StackEdit.

Advertisements

How to Change View of Coding Block in WordPress?

Problem

Many times a good WordPress theme does not render code properly. There are problems like too big font making it harder to read the text. I use StackEdit to prepare posts and publish to my blog.

Solution

There is a way to solve this problem with custom CSS.

Description

The posting source is in Markdown. StackEdit translates MarkDown to HTML and posts to the destination. There is not much I could do with the Markdown. However, the theme on the WordPress blog allows me to set a CSS for particular content.
The code blocks are translated to pre blocks in the HTML. So adding a custom rendering for pre block should fix the problem.
I found a CSS snippet from https://wp-mix.com/css-style-pre-tags/.

pre {
	box-sizing: border-box;
	width: 100%;
	padding: 0;
	margin: 0;
	overflow: auto;
	overflow-y: hidden;
	font-size: 12px;
	line-height: 20px;
	background: #efefef;
	border: 1px solid #777;
	background: url('lines.png') repeat 0 0;
	padding: 10px;
	color: #333;
}

Another good reference is https://perishablepress.com/perfect-pre-tags/

This made the code look much much better.

Written with StackEdit.

How to Pick a Commit of Git Submodule?

Current State

There is a new submodule ‘test’ and we want to add it to the repo.

Steps

Git add submodule

$ cd my_repo
$ git submodule add git@github.com:vishalkanaujia/dev-tools.git

Initialize the repo

$ git submodule update --init --recursive

cd to the submodule

$ cd dev-tools

By default, the latest version is picked. We can checkout to a commit for moving a different version.

$ git checkout abdcca6

Written with StackEdit.

The Best Go Library for Redis

There are two popular choices:

  • go-redis
  • redigo

I prefer redigo over go-redis.

Cons with go-redis

  • Not enough documentation for APIs
  • Difficult to understand naming conventions for APIs
  • I’m still finding how to refresh expiry time of a key using go-redis
  • Needs Cmdable interface for calling Redis commands. The interface is not so well designed.

An interface in go-redis

   type Cmdable interface {
   TTL(key string) *DurationCmd
   Restore(key string, ttl time.Duration, value string) *StatusCmd

Pros with redigo

  • Easy to understand APIs
  • Ability to call Redis commands from the client so you never have to learn a new wrapper
  • No extra interface like go-redis

A sample client with go-redis is as following:

  package main

  import (
      "fmt"

      "github.com/go-redis/redis"
  )

  func HandleErr(err error) {
      if err != nil {
          panic(err)
      }
  }

  /*
  type DurationCmd struct {
      baseCmd

      val       time.Duration
      precision time.Duration
  }
  */

  func ExampleNewClient() {
      client := redis.NewClient(&redis.Options{
          Addr:     "localhost:6379",
          Password: "", // no password set
          DB:       0,  // use default DB
      })
      
      err = client.Set("key", "value", 0).Err()
      HandleErr(err)

      val, err := client.Get("key").Result()
      HandleErr(err)
      fmt.Println("key", val)

      ttl := client.TTL("key")
      fmt.Println("ttl", ttl)
  }

  func main() {
      ExampleNewClient()
  }

Simplifying go-kit toolkit for Microservices – Part I

Introduction

go-kit is one of the most complete and flexible toolkits for developing microservices in Go language. At the same time, the learning curve of go-kit is steep. In this post, I’m trying to explain go-kit fundamental components using general purpose client-server model of Linux.

General Purpose Client-Server Architecture

A server in Linux binds & listens on a port, accepts connections and server requests. A client creates its own socket, makes a request to the server (IP: PORT). The request is just a buffer. The server spawns a thread or runs the handler in the same process. The handler is a function that understands request parameters and typecast it to its expected type.
The client receives a response from the server. The response is read into a buffer and then interpreted.

go-kit in Client-Server Paradigm

go-kit helps create a service. A service listens on a well-known port and IP. It is the server equivalent of Linux server. The server defines handlers for different types of request (PUT, GET, POST). A handler is a function called for a service request. The handler is a generic wrapper over service specific business logic implementation. A handler for GET call may eventually call get_status_reservation() in a sample reservation system.

go-kit intends to keep the core logic away from go-kit influence. The core logic (set of functions that your service implements) stays in a file called service.go. The remaining go-kit code tries to access these functions in an abstract manner. There are entities called endpoints, transport and server. Each of these allows a generic interaction of service functions through HTTP (or gRPC).

The overall objective is to expose service functionality through go-kit toolchain.

Transport

It defines the send and receiver buffer structure. Each service API could expect and send different data. The transport defines structures that define request and response for each API

In the next post, I will share endpoints, the most interesting part of go-kit.

Install Redis on Mac OS via Homebrew

Install Redis on Mac OS via Homebrew

  • Install redis package
    $ brew install redis
    Updating Homebrew...
    ==> Auto-updated Homebrew!
    
  • After installation, you’d see the following message:
     ==> Downloading https://homebrew.bintray.com/bottles/redis-5.0.4.mojave.bottle.tar.gz
     ############################################################# 100.0%
    ==> Pouring redis-5.0.4.mojave.bottle.tar.gz
    ==> Caveats
    To have launchd start redis now and restart at login:
      brew services start redis
    Or, if you don't want/need a background service you can just run:
    redis-server /usr/local/etc/redis.conf
    
  • Run redis as following:
    $ redis-server /usr/local/etc/redis.conf
    oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
    

Connect to Redis with redis-cli

redis-cli 5.0.4

Usage: redis-cli [OPTIONS] [cmd [arg [arg ...]]]
  -h <hostname>      Server hostname (default: 127.0.0.1).
  -p <port>          Server port (default: 6379).
    -n <db>            Database number.
$ redis-cli -h <host> -p <port> -n <db>

Find a String in Files Excluding Some Files


tags: ‘shell, linux, command, development, grep,find’
categories: development

I use the following command to find a string in a Go repo. The vendor directory is not needed, so skip it.

$ find . -type f |grep -v vendor|cut -d":" -f2|xargs grep -n my_search_string

The command find all files in current directory. Next, it removes all files that have vendor in their path.

401:./.git/HEAD
402:./.git/info/exclude
403:./.git/logs/HEAD

The cut command picks the filename and passed to xargs. Each file is then processed by grep for the search string.