Random thoughts of a warped mind…

September 17, 2014

Reconfig for service discovery

Filed under: Amazon EC2,Chef,Git,Linux,Redhat OpenShift,Ruby,Virtualization — Srinivas @ 09:49

Imagine a constantly changing fleet of servers… New servers (or virtual machines) going online as capacity is needed or being taken offline as load drops… Maybe you already use Chef/Puppet to bootstrap your servers and use their node attributes to populate N servers in a load balancer (Haproxy for example) configuration. 

Now adding/removing a new server (a backend, maybe a rails/tomcat server, whatever) would mean that chef-client has to run on all the haproxy boxes so they know about the new backend (or the one that went away). This would work if you  run chef-client every 5 minutes or so? But why? Chef/Puppet are primarily meant to bootstrap your servers and not to sync state. Enter Reconfig and service discovery.


August 4, 2014

PubKey for SSH public key setup

Filed under: Amazon EC2,Chef,Linux,Redhat OpenShift,Virtualization — Srinivas @ 13:13

Built and started using PubKey for managing user SSH public keys (add, update and revoke access) on my personal EC2 and Google compute fleet… Try it out – https://www.pubkey.in/console/ . Docs available on http://docs.pubkey.in and for you lazy sysads, there is a Chef cookbook available too from https://github.com/onepowerltd/pkagent_cookbook :-)



May 30, 2013

Source config info from a Chef data bag in a knife executable script

Filed under: Amazon EC2,Chef,EC2,Ruby — Srinivas @ 03:59

Its nice not to have to hardcode config info into multiple scripts – A clean way to do this is to use Knife data bags . This way config information can be shared between cookbooks run on chef managed instances as well as used in one-off scripts run from your chef management box.

The trick is to use ‘Chef::DataBagItem’ in your script and execute this script via “knife exec scriptname.rb”. This means that the execution of this script will use your chef login info to access your chef server, access the data bag and pull the keys from it into your local script.


March 8, 2013

My chef cookbook to provision EBS PIOPS volumes at Amazon EC2

Filed under: Amazon EC2,Chef,EC2,Git,Linux,Ruby — Srinivas @ 20:21

Heres my Chef cookbook to provision PIOPS EBS Volumes at Amazon EC2 (PIOPS volumes can be requested with a specific IOPs count which is guaranteed to be provided, something plain vanilla EBS volumes dont/cant do).

This cookbook is intended to be a drop-in replacement for Opscodes standard AWS cookbook (except that the namespace used for the node attributes is different, so a little manual work/scripting would be required). Support for Elastic volumes and Load balancers coming shortly…

This cookbook uses the Fog gem so is more easily extendable (or extensible?) to add more AWS resources as needed… Fog v1.6.0 or higher will be auto-installed when this cookbook is invoked.

See Onepower_AWS  for details.

Things to note -

1. Not all AWS Availability zones support PIOPS volumes, see bin/test_piops_support.rb for figuring out which AZs support PIOPS volumes.

2. PIOPS volumes tend to cost more than plain vanilla EBS volumes – And for a real performance gain, PIOPS volumes must be used with instances launched with the ebs-optimized flag as thats what sets up instance specific QoS at AWS to sort of segregate dedicated I/O bandwidth for the instance to access a PIOPS volume (separate from the standard network interface bandwidth that normal EBS uses/shares).

August 15, 2012

Making Chef servers GUI and API endpoints accessible over SSL

Filed under: All,Amazon EC2,Chef,Development,Linux — Srinivas @ 14:34

If you run your own Chef server instance at EC2, its probably in a specific security group that other hosts in the same group can access. However, you may not want to have instances in other regions access this Chef server via unencrypted connections – say you have the chef server in east coast and you have servers in west coast that need to talk to it…

In this case, Its handy to spin up Nginx on the chef server box and have it reverse proxy access over SSL to the Chef servers ports i.e. TCP/4000 for the REST API and TCP/4040 for the Web GUI.


Older Posts »

Powered by WordPress