Caching pip packages using devpi & docker

If you’re using python packages & virtualenvs a lot, you can drastically speed up the time to setup your environment, (which mostly involves download of pip packages, which tend to be network intensive & time consuming) by caching pip packages. Similar to other caching proxies like apt, python has its own caching proxy in the form of devpi, which allows you to run a pypi mirror in your laptop. (devpi is much more than just a pip mirror, for more on its capabilities read the link) »

Collecting distributed system links

Of late I’ve been trying to learn a lot about distributed systems. Also started a github project awesome distributed systems, which hopefully will grow over time to collect links for readings in this domain. If you know of any links please consider adding to the project. »

The very basics of service discovery

This is meant to be an introductory post to a series of posts about Service Discovery and Distributed Consensus. Hopefully this will be series of posts on more aspects of service discovery & Consul etc. Service Discovery We begin with the simple problem of identifying which host/port services are running. For eg. in a small webapp, a simple static conf file could point to the DB node. As functionality grows you would have various (preferably stateless) services talking on different ports (and even different servers). »


Moved my blog from pelican to Hugo. Though there was nothing wrong with pelican as such, mostly I found a problem with my blog/theme not as update with pelican upstream. Hugo is a static site generator written by Steve Francia (@spf13). It is written in Go & amazingly fast to generate a site. Developing sites with hugo is kind of fun; running hugo in watch mode, you get to see the site as you write. »

Running Go ci tests in a docker

I’ve been toying around with the idea of using containers for running ci tests, primarily to have a quicker feedback loop; instead of setups that involve VMs etc. They are ideal for getting an environment up and running quickly and cheap to throw away too. A project which I’m spending some time lately go-ceph, which provides Go bindings for ceph/rados, kind of ideally fit the bill for using this, since testing this project locally, usually needed something like a VM running a ceph cluster, or a locally running ceph. »