Tag: storage

etcd: A Reliable Distributed KV Store

Pronounced et-cee-dee Persistent KV store Uses Raft protocol to maintain data consistency Developed in Go Popular Use case: Kubernetes, Consul Installation (OSX) Step 1 $ ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)” < /dev/null 2> /dev/null Step 2 $ brew install etcd Step 3 $ which etcd /usr/local/bin/etcd Start Server and Make Requests Step 4 $ /usr/local/bin/etcd [WARNING] Deprecated ‘–logger=capnslog’ flag is…

What is a Data Platform?

Over time, organizations need to go beyond a single DB for querying and storing data to a set of DBs that cater to different business requirements. A Data Platform might comprise: Search Index A relational DB NoSQL DB Data Warehouse Why a Data Warehouse? It is a subject of interest to understand how the application uses the DB. The inspection…

The most efficient Image format- WebP?

The most efficient Image format- WebP? I always knew that PNG followed by JPEG were the most efficnent, lossy image compression format. However, there is a new image format, webP. It is developed by Google and comes in lossy as well as lossless format. How is it better than PNG/JPEG? The webP image is ~30% smaller in storage size than…

Hard drives: Native Command Ordering

A simple hard drive today is capable of things that sound like some outlandish technology. Just try to do some file I/O in your application and do it with many threads. Say you have 4 threads, A,B,C, and D. And request to do I/O comes in A then B and so on. If you check the return status of these…