A Quick Look at Statsd

The idea of application performance monitoring with minimal side-effects brought Statsd to life. It is a daemon service that listens to UDP packets. It expects text payload, aggregates the metrics, and send them over to a metrics server (a time series DB).

The metrics data is flushed at regular intervals (default 10 seconds). After every flush, the counter metrics reset to zero whereas gauge metrics send the previous value.

The metrics and operations are of the following types:

  1. Counter – Increment & Decrement
  2. Gauge – A fixed value

The Development language is primarily JavaScript.

Reference

Written with StackEdit.


K-Messed Array Sort: Python Solution

K-Messed Array Sort

Given an array of integers arr where each element is at most k places away from its sorted position

Solution

from Queue import PriorityQueue


def sort_k_messed_array(arr, k):
  kheap = PriorityQueue()
  result = []
  
  for num in arr:
    kheap.put(num, 'num')
    
    if kheap.qsize() == (k + 1):
      item = kheap.get()
      result.append(item)
    
  while not kheap.empty():
    result.append(kheap.get())
    
  return result

arr = [1, 4, 5, 2, 3, 7, 8, 6, 10, 9]
print(sort_k_messed_array(arr, 2))

Result

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Reference


What is Special in HTTP 2.0?

Useful Points:

  1. Unlike text protocol HTTP/1.0 & HTTP/1.1, HTTP/2.0 is binary based protocol. So tools like curl are needed to make requests instead of plain telnet.
  2. The TCP connection is now multiplexed for multiple requests. So a client should open only one connection.
  3. Few browsers support HTTP/2.0 with only TLS.
  4. The compression of data is not encouraged.

Reference


go-kit: Sample Code for Function to Log

go-kit: Sample Code for Function to Log

func getLogger() log.Logger {
    var logger log.Logger
    logger = log.NewLogfmtLogger(log.NewSyncWriter(os.Stderr))
    logger = log.With(logger, "instance_id", 123)
    return logger
}

The above function is easily usable in a go-kit project to log JSON style data.

func myfun() {
    mylogger := getLogger()
    mylogger.Log("a key", "a value")
}

Reference

Written with StackEdit.


Reactive Extensions: Asynchronous Reactive Systems

Reactive Extensions: Asynchronous Reactive Systems

Reactive Extension is a paradigm to develop systems using the react model. It’s an event-driven state machine for functions. It’s possible to represent either synchronous or asynchronous communication with Reactive Extensions.

All we need is a message broker, producer of events (Observable), and consumers(Observer).

Typically, the consumer processes an event with a set of transformation/ processing. Each of the transformations is a step in the reactive pipeline. Each stage processes the data from the previous state.

In a simple sense, a typical function flow in a listener component is as following:

1. Get message
2. Unpack message
3. Process the payload
4. Log the result

With reactive experience, each step can become a thread, having a pipe for reading/writing data. Each step can be synchronous or asynchronous. We can have many pipelines, defined with a different set of tools (filters, map).

Reference


HTTP Accelerator Cache: Varnish

HTTP Accelerator Cache: Varnish

The Varnish is an open-source reverse proxy server. Its primary purpose is to cache HTTP responses. Besides it also enables modifying HTTP requests/ response to the origin.

The Varnish documentation highlights that the server is highly scalable on a multi-core system, efficient (uses mostly kernel services such as kqueue, epoll, memory management). It’s developed in C.

The reverse proxy sits between the client (browser) and the server (origin).

It has a few interesting use cases such as:

  • Modifying cookies in the request
  • Appending/removing headers in request/response (e.g. changing Host header)
  • Prevents Cache Stampede (on a miss, many fetches won’t cause many reads from the origin). It uses TTL + Grace period for cached objects. After TTL is expired, Varnish fetches the object asynchronously.

References

Written with StackEdit.


Regex to find 6 repeating characters

Duncan's blog

I wanted to check our codebase for where we had six repeating characters in HTML and CSS for colours, e.g. #FFFFFF for white.  Here’s the regular expression for it, basically blogged here so I can remember where to find it later:

([a-fA-F0-9])1{5}

So [a-fA-F0-9] looks for a single character that is one of the letters A – F (only need to go up to F because we’re dealing with hexadecimal numbers), or the digits 0 – 9.

The ( ) parentheses around that turns it into a backreference.

The 1 then refers to that backreferenced matched, and the {5} says to match it exactly 5 times.

So it’ll find where there’s a single matching character that is then repeated five times.

What I was doing this for was to replace all six character codes (that repeat) for colours with three character codes (because #FFF is equivalent to #FFFFFF). So to…

View original post 92 more words


Discussion on Microservices Integration

Discussion on Microservices Integration

Microservices is a distributed system pattern and components need to communicate through the network.

If communication data includes internal technical details of participating components, the system loses the property of loose-coupling.

If a change in one service causes changes in many other services, we lose the property of high-cohesion.

An unreliable network and added latency for data communication dictate choices for services integration.

There are two major types:

  • Direct Communication
  • Event-based, asynchronous communication

Direct communication is a request/response-based. It is useful for low-latency and immediate consumption scenarios. It is prone to failures (server unresponsive, network latency) and hence caller/clients need to retry. This communication is not easily extensible and soon become brittle.

Asynchronous mode is a “fire and forget” approach. An event is generated and it is up to the consumers to handle it. This model scales very well: the publisher and consumers have no coupling. Both are independently deployable. However, it is hard to monitor the status of each event handler.

References

Written with StackEdit.


Domain Driven Design: Bounded Context

Microservice architecture borrows many classical concepts. Bounded Context is one of the most important ideas of domain-driven design.

A context means specific responsibility. Bounded Context implies a responsibility with explicit boundaries.

Reading from a few blogs, the bounded context means more of real-world identification of responsibilities. It’s seeing things from different altitudes: An application is a bounded context. A level below, it’s UI, backend are BC.

Overall, seeing a BC as a cell (like an organic cell) came more naturally. A cell defines a specific set of responsibilities. It has an explicit interface to accept/ reject model (communication data) requests.

Why Bother Bounded Context

  • BC promotes loses coupling and high cohesion.

References

Written with StackEdit.


How network work in docker-compose?

How network works in docker-compose?

docker-compose creates a set of containers, brings them up along with a private network. The network is very interesting because a private network is essential to compose & test a microservice bases system.

I found the post https://medium.com/@caysever/docker-compose-network-b86e424fad82 quite useful.

Post Highlights

  • docker-compose creates a default network, if no network is specified.
  • The name of the network is <local dir>_default.
  • The default network type is bridge.
Bridge Network

A bridge is a layer 2 component. It understands segments and uses a hardware/software switch/bridge.

Host Network

Use the host network namespace. It is preferred for achieving high performance. It works on Linux only.

References-