Pre-Deployment Checklist for Developers

  1. Handled all error conditions?
    Have you checked each error and handled or propagated upstream?

  2. Returned appropriate error code
    Please return proper error code & detailed error message to callers.

    msg := fmt.Printf("[fname] failed in Redis Set err=%s ret=%d", err, ret)
    ret errRedis
    
  3. Added logs for each failure
    It will help you narrow down the trouble code.

  4. Successful build with lint
    lint keeps your code clean.

  5. Formatting fixes (gofmt)
    Always follow the coding standard of the language, especially symbol names.

  6. Code reviews by at least two people.

  7. Unit Testing (docker compose)
    UT is easy and catches stupid bugs. There are many ways to test the code. The easiest is just the functions you’ve added. Stub any call you don’t or can’t test.

  8. Scale testing (for finding a resource such as sockets, memory leaks)
    UT is ineffective catching serial and slow bugs. These are the most troublesome problems that can be caught using heavy load & volume testing

  9. Staging deployment

  10. Finally, live on production 🙂

Advertisements

Introduction to Terraform

Concepts

  1. Terraform declares and defines infrastructure pieces of a system.
  2. It is a workflow manager.
  3. The pieces range from IAAS (Azure EC2 instances) to PAAS (Heroku, Kubernetes services) to SAAS (LB, DNS).
  4. It uses providers for each type of component. A provider defines a set of APIs that Terraform calls to create infrastructure.
  5. Since infrastructure is defined in a text file (JSON or HCL format), it is easy to version control.
  6. A TF file is essentially a DAG, that creates components in order.
  7. TF tries to maintain a desired state of the system.
  8. Terraform uses a version control solution such as CVS, SVN, or Git.
  9. Hence it is easy to track changes and manage simultaneous multiple changes.
  10. Terraform is mostly used for creating infrastructure, not configuring it.
  11. There are four basic commands in TF:
    • Refresh
    • Plan
    • Apply
    • Delete
  12. Terraform is coded primarily in Golang.

References

How to Select a Random Key from a Hash Map in Constant Time?

Premise

A hashmap has a time complexity of O(1) for all operations.

Problem

You have to find a constant time method to uniformly random selection of a key in a Hash map.

Assumptions

  1. The map can grow to memory size.
  2. You can use any readymade hash map.

Solution

I’m discussing the pseudo code for a Python solution. All operations work in constant time except remove().

randMap = {}

def insert(k, v):
    randMap[k] = v    

The random must pick a random key. But how? We can store all the keys in a list and then run random() on the list indexes.

randMap = {}
indexMap = {}
keyList = []
last = 0
def insert(k, v):
    randMap[k] = v
    keyList.append(k)
    last += 1
    
    # Maintain index of each key
    indexMap[k] = len(keyList)    
def getRandom():
    start = 0
    end = len(keyList) - 1
    randomIndex = random(start, end)
    key = keyList[randomIndex]
    
    return key

How do you delete a key?

This is the crux of the problem. Deletion of a key would need us to change the index array too. That means we need to shift all elements of the array and complexity would shoot to O(n).

def remove(k):
    # Remove from the hash map
    randMap.pop(k, None)
    
    # Remove from key list
    index = indexMap[k]
    
    # This is O(n) operation
    keyList.remove(index)

    # Pop the index too
    indexMap.pop(k) 

How to improve deletion?

  1. Delete the entry and mark the entry invalid. The probability distribution does not change, however with more and more deletions, you have a sparse array. Thus you would need multiple getRandom() to get a valid key.
  2. Move the last element to the deleted element. Adjust the index of the last element.
def remove(k):
    # Remove from the hash map
    randMap.pop(k, None)

    index = indexMap[k]
    indexMap[last] = index
    
    # This is O(1) operation
    keyList[index] = keyLast[last]

    # Pop the index too
    indexMap.pop(k)
    last -= 1 

This is a constant time solution.

How to Write a Conference Scheduler

The conference schedule is an NP problem since it is O(2**n). A greedy approach is pragmatic to solve the problem. We can pick various strategies to prepare a schedule such as the biggest talk first, smallest talk first, random selection, etc.

The Python code uses the Longest talk first and random selection to prepare schedules.

Github Code

https://github.com/vishalkanaujia/dev-tools/tree/master/conference_scheduler

Written with StackEdit.

Internals of Linux Process Signals

Linux Signals

  • A process in Linux is represented as task_struct.
  • A process A sends a signal to process B using system call kill() or kill -<sig num> <pid>.
  • Kernel updates the task_struct of the receiving process B. It changes the signal field with the passed value.
  • The scheduler checks the signals on a process before running the process. If the signal is set, it calls the signal handler.
  • A process can define its handlers for all signals except SIGKILL and SIGSTOP.
  • There is always a default signal handler for each signal.

References