Infrastructure as Code IaC is the process of managing, provisioning and configuring computing infrastructure using machine-processable definition files or templates. They permit system architects, engineers, and developers to leverage reusable and updatables templates to deploy and manage complete resource stacks. As the stack is templated, detailed and versioned, using such tools results in efficient and fast deployment to serve modern DevOps methods such as continuous delivery.
Automation of the delivery using these tools also eliminates risks of misconfigurations and together with automated testing protects production environments. In this article we will delve deep into CloudFormation, discussing the formats and structure of templates using real-life examples. Discover more about how to obtain versatility with Infrastructure as Code here.
Take a look at its basic components:. All the parameters we want to be editable in our stack. We can use them to introduce passwords, name a particular resource, or decide the type of resources, as shown in these examples:. Deciding which IP address can access our resources :. The example below shows the mapping of an advanced configuration. In addition, it will deploy an ElastiCache nodes across multiple availability zones:. Conditions are generally used to match parameter values with resources or properties.
For example, using the Parameters at , we can match the application branch to different Security Groups :. These Conditions can also trigger choices in the Resources section. The Resources section contains all the AWS objects we want to include in our stack. Continuing from the preceding example, we can match a Security Group to the application branch assuming we want our staging instances available only to the CustomerIpAddress declared in the Parameters :. The Output section collects the information that we may need or want to pass to other stacks.
This gives us smaller and more readable files, the ability to add comments, and support for multi-line strings. Managing CloudFormation templates has recently become easier: AWS has introduced a graphic tool with diagrams and a drag-and-drop interface for managing templates. Creating, viewing, and updating even a complete stack is fast and less prone to errors as we have a complete overview of all the resource correlations while editing the details.
Templates can be edited using a canvas pane or using a simple text file, but pay attention, currently only JSON formatted templates are supported.
As mentioned above, key advantages of IaC tooling include life cycle management and derivative ownership. In addition, each party e. Another important tool to to mention CloudFormer which allow you to create a new template based on your existing cloud resource. The template configuration is stored in S3 and using CloudFormer you can create a template of a stack of an application that is already running, allowing you to replicate it in the same AWS region or even across regions. Maintenance of the CloudFormation stacks can also be a challenge and updates of these can also be a mess.
It was hard to accurately grasp all the implications of the changes we wanted to introduce, especially over running resources. Another interesting feature of CloudFormation is the option to rollback a failed update.
Rollbacks are used even during the first creation of the stack, and sometimes we have to disable them to permit a better debug of a failing stack. Getting more and more mature, CloudFormation is gaining more supporters and users are creating tools to make easier to create and manage templates and stacks.
CloudFormation is one of the key elements that create the great Amazon cloud flexibility. However it fits AWS only.
For this reason, Terraform deserves a paragraph on its own. Terraform can manage existing and popular service providers as well as custom in-house solutions. Learn how to deploy Terraform on OpenStack workloads here.AWS Cloudformation Parameters tutorial - Beginner tutorial - Cloudformation Series Part 5
In this category we should also mention the configuration management tools such as Chef and Ansible which can also help in provisioning an application stack, however can also complement them. Organization that look to automate deployment and delivery should take a closer look at tools such as CloudFormation and Terraform as these type of solutions are fundamental when looking to build a robust DevOps environment.Topics include:.
This greatly improved string concatenation in CloudFormation. Previously if you needed to append strings together, you had to use the clumsy Fn::Join syntax. This was barely readable, and required a lot of effort to parse and understand what this was trying to do. This is much more readable! The abbreviation of the intrinsic functions denoted by the leading! CloudFormation intrinsic functions have two different forms, the standard form, and a tag abbreviation.
Subscribe to RSS
The only limitation is that you cannot nest additional functions in the abbreviated tag. For example you cannot import a value inside the abbreviated version. The Fn::Sub syntax has two very different forms. Above we saw the short form. Only simple parameters or resources can be included inside the string argument. This allows you to reference values from other CloudFormation stacks without having to tediously pass them in as Parameters.
Say you have a common CloudFormation template which establishes a Route53 hosted zone for you. All future CloudFormation stacks can reference an exported value from this stack using the! ImportValue function. Our first template creates a Route53 hosted zone. Note that the difference between the zone name and the DNS name is the trailing period required for zone names. Which we append using the Sub short form. These values can be referenced in subsequent CloudFormation stacks using the ImportValue function:.
Here you can see a more complex version of the Fn::Sub syntax, where the argument is an array of two elements. Here the parameter name HostedZoneName is not passed into the template as a parameter, nor is it the name of a resource created elsewhere in this template. It is a temporary parameter that exists only in the scope of this Fn::Sub function. Its value is supplied in the map of the second argument:. Note: It is important to see that the two map values, HostedZoneName and SubDomain are elements of a map, and it is that map which is the second array element.
This is where YAML syntax gets a little odd. If this were a function call, the arguments might look something like this:. If we needed to pass in the name of our stack, we would have to include additional Sub functions to put together our import values:. This is the name of the CloudFormation stack that was deployed earlier with our Route53 zone and exported values. This is done by the innermost!
General Template Snippets
Sub expression:. This expression concatenates the stack name parameter with our naming convention. This resolves to the export parameter name, which can then be used by the Fn::ImportValue function above it. This in turn is used as the value for the ZoneName local parameter, which in turn is used in the outermost Fn::Sub function to finally piece together our desired DNS name.
Another really useful place to use the Fn::Sub function is when you need to have long blocks of text interspersed with parameters. In this example, we want to install the EC2 Systems Manager agent on a host at deployment time.
The first new thing you might notice is the pipe symbol following the!If you've got a moment, please tell us what we did right so we can do more of it. Thanks for letting us know this page needs work. We're sorry we let you down. If you've got a moment, please tell us how we can make the documentation better. The intrinsic function Fn::Sub substitutes variables in an input string with values that you specify.
In your templates, you can use this function to construct commands or outputs that include values that aren't available until you create or update a stack. If you're substituting only template parameters, resource logical IDs, or resource attributes in the String parameter, don't specify a variable map.
Variables can be template parameter names, resource logical IDs, resource attributes, or a variable in a key-value map. If you specify only template parameter names, resource logical IDs, and resource attributes, don't specify a key-value map. The name of a variable that you included in the String parameter. AWS CloudFormation returns the original string, substituting the values for all of the variables.
For readability, the JSON example uses the Fn::Join function to separate each command, instead of specifying the entire user data script in a single string value. For the String parameter, you cannot use any functions.
Document Conventions.This is something the AWS community has been begging for, for quite a while. A couple of things you notice in this example are how clean the code looks and the comments. These are both necessary to make code descriptive and clear. When I removed them comment 1 and 2the CFT still worked. Additionally, as you will notice in my second set of comments, I switch from 2-space to 1-space YAML formatting comments 3 and 6.
Inline with the hyphen and the rest following after with the same spacing comment 4. Multiple AWS examples do it in different ways. This makes the creation of userdata scripts very close to the actual script you would run on the server. The new! Of course if you miss it, the Fn::Join can still be used like this:. Notice that nested lists are created using two hyphens. This is a big advantage over tools like Terraform where all new templates would have to be built from scratch, particularly since a conversion tool could probably be whipped together in short order.
If you continue to use this site we will assume that you are happy with it.The Quick Start team has developed boilerplates for the Quick Start master and workload templates. You can find these in the GitHub quickstart-examples repository.
As discussed in the Modularity section, you will need both master and workload templates for your Quick Start. The quickstart-examples repository also includes code for integrating AWS services, such as adding an Amazon Redshift cluster to your Quick Start. We also recommend that you review the following templates and use them as building blocks for your Quick Start. These are referenced in the master and workload template boilerplates. A second, optional set of private subnets includes dedicated custom network ACLs per subnet.
Adding bastion host functionality for secure Linux deployments — These templates deploy Linux bastion hosts that provide secure access to your Linux instances in public or private subnets. Additionally, the Quick Start deployment provides custom resources that enable you to deploy and manage Kubernetes applications using AWS CloudFormation.
The dark mode beta is finally here. Change your preferences any time. Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information.
In this example from AWS docs we have a UserData field that allows a multiline string, using the following syntax:. What does! Sub mean here, especially the pipe character? Sub is used. Sub substitutes variables in an input string with values that you specify.
In your templates, you can use this function to construct commands or outputs that include values that aren't available until you create or update a stack.
The character ' ' pipe symbol means "Literal Style". This uses a simpler, more readable scalar style. In your UserData section the! In addition the pipe symbol at the end of a line in YAML signifies that any indented text that follows after the!
Sub should be interpreted as a multi-line scalar value. For more details refer, intrinsic function details documentation. Learn more.
Ask Question. Asked 2 years, 5 months ago. Active 2 years, 5 months ago. Viewed 7k times. Is the pipe character standing for a newline, saying lines must joined by newlines? Active Oldest Votes. John Hanley John Hanley Ashan Ashan Sign up or log in Sign up using Google.
Sign up using Facebook. Sign up using Email and Password.
Nested Sub functions
The dark mode beta is finally here. Change your preferences any time.
Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information. Now in my main template I want to refer to nested template presented above and pass a bash script using the userData parameter. Additionally I do not want to inline the content of user data script because I want to reuse it for few ec2 instances so I do not want to duplicate the script each time I declare ec2 instance in my main template.
All works ok, if I paste my script directly into to the parameters section as inline script when calling my nested template instead of reffering to string set as parameter :.
Here are a few options on how to reuse a bash script in user-data for multiple EC2 instances defined through CloudFormation:. Then, in your nested stack, apply any required intrinsic functions Fn::Base64as well as Fn::Sub which is quite helpful if you need to apply any Ref or Fn::GetAtt functions within your user-data script within the EC2 instance's resource properties:.
You can upload your single Bash script to an S3 bucket, then invoke the script by adding a minimal user-data script in each EC2 instance in your template:. This approach will allow you to eliminate duplication in your source files - although the templates will contain 'duplicate' user-data scripts, this will only occur in the generated templates, so should not pose a problem.
You'll have to look outside the template to provide the same user data to multiple templates. A common approach here would be to abstract your template one step further, or "template the template". Use the same method to create both templates, and you'll keep them both DRY. I'm a huge fan of cloudformation and use it to create most all my resources, especially for production-bound uses.
But as powerful as it is, it isn't quite turn-key. In addition to creating the template, you'll also have to call the coudformation API to create the stack, and provide a stack name and parameters. Thus, automation around the use of cloudformation is a necessary part of a complete solution.
This automation can be simplistic bash script, for example or sophisticated. I've taken to using ansible's cloudformation module to automate "around" the template, be it creating a template for the template with Jinja, or just providing different sets of parameters to the same reusable template, or doing discovery before the stack is created; whatever ancillary operations are necessary. Some folks really like troposphere for this purpose - if you're a pythonic thinker you might find it to be a good fit.
Once you have automation of any kind handling the stack creation, you'll find it's easy to add steps to make the template itself more dynamic, or assemble multiple stacks from reusable components. At work we use cloudformation quite a bit and are tending these days to prefer a compositional approach, where we define the shared components of the templates we use, and then compose the actual templates from components.
This works OK in simple cases, but the combinatorial complexity of all those conditions tends to make this a difficult solution in the long run, unless the differences are really simple.
Actually I found one more solution than already mentioned. This solution on the one hand is a little "hackish", but on the other hand I found it to be really useful for "bash script" use case and also for other parameters.
The idea is to create an extra stack - " parameters stack " - which will output the values. Since outputs of a stack are not limited to String as it is for default values we can define entire base64 encoded script as a single output from a stack. The drawback is that every stack needs to define at least one resource, so our parameters stack also needs to define at least one resource.
The solution for this issue is either to define the parameters in another template which already defines existing resource, or create a "fake resource" which will never be created becasue of a Condition which will never be satisified. Here I present the solution with fake resource.
First we create our new paramaters-stack.