In the past few weeks, AWS has released into preview two exciting new services. At our disposal, we now have Amazon Managed Service for Prometheus (AMP), a Prometheus compatible managed monitoring solution for storing and querying metrics at scale. Additionally, we now also have Amazon Managed Service for Grafana (AMG), which as you would expect, is a fully managed data visualisation service with logically isolated servers. AWS handles all the provisioning, scaling and security automatically.

AMG (Grafana) is currently in limited preview, so I got myself access to it and have had a play to see how both of these new managed services work together for some of the standard ways I see customers using AWS.

It’s worth considering that both of these products are in preview and therefore have limited availability to certain regions. For AMG (Grafana) this is only us-east-1 and eu-west-1...

I recently published a video, and one of my first talking-head style videos, on a quick introduction to information security.

Here’s the original script I wrote to help me record the video.

Script

Hey folks!

I wanted to talk security as part of application development today, as one of my favourite questions to ask in full-stack engineer interviews is about broad security understanding, and I often don’t get great replies.

Security for me is everyone’s responsibility, while your company may a dedicated security team of some sort, the people who are best placed to secure an application or platform, are the people working on it every day.

In this series of videos, I’ll dig a little deeper into security as a whole, and what we can do as engineers and leaders, to help continuously improve it.

My background is as a software engineer, and I’m now leading teams to improve and...

I recently published a demo of Hashicorp Vault’s dynamic secret feature.

You can follow along with the demo using all the commands here: https://github.com/florx/secrets-are-hard-demo

If you’re more of a text person than a video person - here’s what I covered and why they’re so cool.

What are they?

Dynamic secrets are credentials that don’t exist until you request them. Once you request access to a specific system (e.g. a database), the centralised secret management system (in this case Vault) will generate credentials and serve them.

These credentials will be time-limited, and if the requesting system doesn’t check in to say it’s still using them, they will be automatically revoked. This whole process of generation, leasing, and revoking, means any secrets are only available for a very short time and are only kept in memory.

Why is it important?

Static secrets, the more...

Thinking about shutting down your non-production environments at night, but wondering about the benefits?

Before we started shutting down at night, we were spending a small fortune on hosting environments that simply weren’t used during nights and weekends. Plus we didn’t have a good recovery strategy plan, if our infrastructure failed - so we decided to combine the two, and solve with one.

I’ll talk about what it means, the benefits and how to get started, in your own adventure of shutting down non-prod at night.

What do I mean, by nightly shutdowns?

By a nightly shutdowns, I actually mean terminating all of your non-production cloud servers at a specified time (e.g. 9pm) and then starting new instances again in the morning (7am). You can also leave these environments “off” over the weekend too, by simply not starting the instances on the Saturday and Sunday mornings.

Are you

I have an unapologetic and relentless obsession to automation. It drives the vast majority of my decision making, both in personal and work contexts, and also the culture of the teams I build.

This is a story of how I identified three manual pain points on a project, and completely automated them away. This eradicated both human error, and human laziness. I’ve often found the laziest engineers make for the best engineers, as they’ll find the simplest solution - and if repetitive - automate the task so they only have to do it once.

We aim to automate every (sensible) aspect of our systems, so engineers don’t have to read docs to remember how to get a service to production - the pipeline/code just makes it happen.

Automation Problem 1: Not all repos have the same settings

Once you grow past only a handful of repositories, it becomes problematic to keep all the settings the same, and...

I recently attended a conference, designed to bring together security and engineering teams, I was struck by how many people use the word DevOps in different ways, and to me - badly, wrongly and out of context.

“DevOps” used badly

• I heard people ask, are you DevOps?
• Do you do DevOps?
• Recruiters hiring for a DevOps engineer

What does DevOps mean to me?

DevOps is modern, cultural approach to engineering. To understand this more, we need understand historically what we did before this idea came to be.

Traditionally engineering teams were split, into those who wrote code, and those made sure production carried on working. These teams were also known as “Developer” teams and “Operations” teams.

This produced some communication and handoff difficulties between the two styles of teams, as when the Developer team was ready to run something in production, they would “kick the code over...

It’s important in our world to release as often as possible to production. This means we get to move quickly, try new things, and then get results for the business quickly.

One of the issues with moving quickly can be that, with increased automation, there can be more reliance on tools and it’s easy to forget to approve a production deployment.

We use a pipeline that looks something like this:

We’re doing Continuous Integration and Delivery, but not quite Continuous Deployment. We continuously deploy to our lower environments, but not production quite yet. As you might expect, our master branch is deployed to our development and pre-production environment automatically, but can sometimes get stuck at the manual approval step.

In order to check where we’re at, I’ve written and Open Sourced a quick script that will check how far behind production is on each repository. Take a look...

Building distributed systems in using a microservice pattern is hard. At my company we’re always looking for ways to automate any manual processes, or anything that is difficult. Computers don’t make mistakes, but humans aren’t infallible. The more we can rely on a machine-led process, the more reliable a release process can be. This is the journey of versioning and releasing for one of our projects.

In the beginning…

We’re using Kubernetes, with Helm to manage our deployments. We started by using the SHA1 Git hash as the version number, it’s pretty unique and means there’s no manual intervention required by a developer to “bump” a version number. It’s automatically “bumped” by git every time we squash and merge a pull request to master!

Each of those merges to master, triggers an automatic build and push the code to both the dev and preprod environments, it then waits for approval...

Our industry is not shy about inventing new terms to describe what new things they’re doing.

We have DevOps, which is all about the culture shift from the old style of writing some code, then throwing it over the wall for some guy to run in production for you. As the Project Phoenix novel outlines, this rarely works, so DevOps is about moving the development team and the operations team closer together, to improve communication and collaboration, allowing deadlines to be shared rather than fought over.

For me, NoOps takes “moving the dev and ops teams closer” to the next level. Every developer should deploy, and regularly.

So what is NoOps?

For me, it’s important that Developers don’t speak to Ops anymore. That’s because they are the Ops team too. There’s no dedicated Ops person on my teams, each individual developer has the knowledge and access to do deployments as regularly as...