Uncategorized

Notes on RGW Sytem Object State

RGW raw object store has following structure: // rgw/rgw_rados.h struct RGWRawObjState { rgw_raw_obj obj; bool has_attrs{false}; bool exists{false}; uint64_t size{0}; ceph::real_time mtime; uint64_t epoch; bufferlist obj_tag; bool has_data{false}; bufferlist data; bool prefetch_data{false}; uint64_t pg_ver{0}; /* important! don’t forget to update copy constructor */ RGWObjVersionTracker objv_tracker; map<string, bufferlist> attrset; RGWRawObjState() {} Written with StackEdit.

Read more
Uncategorized

Notes on RGW Request Path

The principle class is RGWOp. It defines request state, RGWRados store pointer. A RGW request struct req_state has Ceph contect op type info account, bucket info zonegroup name RGWBucketInfo bucket_info RGWUserInfo *user Op Execution RGWGetObj::execute() is the primary execution context under the class RGWGetObj. It uses interfaces of class RGWRados::Object to perfrom I/O ops. The […]

Read more
Uncategorized

Notes on RGW Manifest

RGW maintains a manifest of each object. The class RGWObjManifest implements the details with object head, tail placement. Manifest is written as XATTRs along with RGWRados::Object::Write::_do_write_meta( ). /** * Write/overwrite an object to the bucket storage. * bucket: the bucket to store the object in * obj: the object name/key * data: the object contents/value […]

Read more
Uncategorized

Notes on Ceph librados Client

Cluster Connection A client is an application that uses librados to connect to a Ceph cluster. It needs a cluster object populatd with cluster info (cluster name, info from ceph.conf) Then the client do a rados_connect and cluster handle is populated. A cluster handle can bind with different pools. ​ Cluster IO context The I/O […]

Read more
Uncategorized

Ceph Luminous build

struct RGWObjectCtx { RGWRados *store; void *user_ctx; RGWObjectCtxImpl<rgw_obj, RGWObjState> obj; RGWObjectCtxImpl<rgw_raw_obj, RGWRawObjState> raw; explicit RGWObjectCtx(RGWRados *_store) : store(_store), user_ctx(NULL), obj(store), raw(store) { } RGWObjectCtx(RGWRados *_store, void *_user_ctx) : store(_store), user_ctx(_user_ctx), obj(store), raw(store) { } }; Written with StackEdit.

Read more
Uncategorized

Building Ceph on Debian Jessie

Checkout Ceph with –recursive Comment code that installs setuptools and sudo ./install-deps.sh ./ceph/do_cmake.sh. Switch to root using sudo su and run the following commands: echo “LC_ALL=en_US.UTF-8” >> /etc/environment echo “en_US.UTF-8 UTF-8” >> /etc/locale.gen echo “LANG=en_US.UTF-8” > /etc/locale.conf locale-gen en_US.UTF-8 Now switch back to your login and create ~/.bash_profile: $ cat ~/.bash_profile export LC_ALL=en_US.UTF-8 export LANG=en_US.UTF-8 […]

Read more
Uncategorized

s3cmd SSL connection error

Problem Statement File “/usr/lib/python2.7/httplib.py”, line 1263, in connect server_hostname=server_hostname) File “/usr/lib/python2.7/ssl.py”, line 363, in wrap_socket _context=self) File “/usr/lib/python2.7/ssl.py”, line 611, in __init__ self.do_handshake() File “/usr/lib/python2.7/ssl.py”, line 840, in do_handshake self._sslobj.do_handshake() error: [Errno 0] Error Environement Debian 9 s3cmd 2.0.1 Solution The problem happens due to SSL issue. To make s3cmd work, we should invoke it […]

Read more