When using Amazon Redshift, distribution style plays an important role in optimising the table design for best performance. In a nutshell, table’s distribution style dictates how the data is distributed across Redshift node and slices. A key objective is to avoid the data redistribution during query execution or runtime. This is accomplished by locating or co-locating the data where it needs to be before the query is executed.

Hundreds of marketing technologies, thousands of tags - effective tag management requires more than just an enterprise tag management solution. In this post, I present you my 10 commandments for effective tag management. I think these 10 simple rules are quite obvious and useful for anyone interested in implementing a more robust and effective tagging regime.

DevOps is a cultural shift with immediate focus on maximising the business value by opting better communication, collaboration and feedback within and across IT development and operation teams. Infrastructure as Code (IaC) is a key element of DevOps philosophy with benefits for both development and operation teams. The term infrastructure as code is sometimes also referred to as programmable infrastructure.

Ever since the Amazon introduced the AWS Lambda service at AWS re:Invent 2014, a variety of new applications for the service has emerged which highlights tremendous potential and traction for the AWS Lambda service. Over last one year, Amazon has been actively working towards integrating other AWS services with AWS Lambda.

Amazon today announced Amazon Elasticsearch Service (Amazon ES) - a fully managed Elasticsearch service which can support your real-time distributed search requirements. I am not sure about others, but for me a this is a big deal and can be a game changer. I have worked with both Splunk and Elasticsearch. On the one hand, Splunk is definitely a superior product and well-packaged solution for operational intelligence.

I have been playing with Apache Drill for quite some time now. In layman’s terms, Apache Drill is SQL query engine which can perform queries against any type of data store - in particular any non-relational data store. Apache Drill’s ability to query against raw data files stored on the cloud storage platforms such as Amazon S3, Azure Blob Storage, Google Cloud Storage makes it a really attractive and cost effective query engine.

We are observing a new era in database technology space, drop-in replacements with order of magnitude improvement in performance over their counterparts. First Amazon Aurora and now Scylla DB. Although the concept of drop-in replacements is not something new but most recently these substitutes started pushing the performance boundaries using completely overhauled architecture.

If you work with AWS CloudFormation on a day-to-day basis, I think you might be interested in visualising your AWS CloudFormation templates. CloudFormation stack visualization can be quite handy for lifecycle management of your AWS deployment. In addition, visualization can be immensely helpful in change management and governance.

Aurora is a new offering from Amazon Relational Database Service (RDS). It is advertised as drop-in replacement for the MySQL - full compatibility with MySQL 5.6. Aurora is designed for fault-tolerance, availability and storage elasticity. It is a relational database and a highly cost effective one.

Currently the majority of cloud based database and data warehouse services are provisioned with fixed storage and compute resources. Resizing of resources cannot be performed without compromising availability and performance. This means service users typically end up with over-provisioned under-utilised expensive resources to accommodate possible peak demand. In the worst case, under-provisioned resources unable to handle sudden work overloads.

In 2005 Stonebraker et al. published a paper that outlined 8 key requirements for stream processing architecture. These key requirements can be easily translated into building blocks of stream processing architecture. Although, this article dates before systems such as Apache Kafka, Amazon Kinesis, Apache Spark, Apache Storm, etc. most of the requirements are still relevant.