Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lint and compile commands #245

Merged
merged 7 commits into from
Jul 19, 2018
Merged

Add lint and compile commands #245

merged 7 commits into from
Jul 19, 2018

Conversation

viraptor
Copy link
Contributor

@viraptor viraptor commented Jul 11, 2018

Make it easy to use cfn-lint on the templated stacks. The templated
stack is written to a temporary file which cfn-lint is run on. Since
the error messages contain the path to the json node, it's not that
important to preserve the rendered json with matching lines.

Unfortunately there's no special error code for success/failure. We can
just forward the original output.

Having cfn-lint is not a requirement for stack_master and the missing
binary is handled without exploding.

Also add a compile command which outputs the processed stack
template for use in other tools.

Make it easy to use cfn-lint on the templated stacks. The templated
stack is written to a temporary file which cfn-lint is run on. Since
the error messages contain the path to the json node, it's not that
important to preserve the rendered json with matching lines.

Unfortunately there's no special error code for success/failure. We can
just forward the original output.

Having cfn-lint is not a requirement for stack_master and the missing
binary is handled without exploding.
@viraptor
Copy link
Contributor Author

Failure:

Executing lint on market-splunk-indexer in us-east-1
Failed to run cfn-lint, do you have it installed and available in $PATH?

Success:

Executing lint on market-splunk-indexer in us-east-1
cfn-lint 0.3.3
E2002 Parameter MarketAppVpcNatGatewayIps has invalid type List<String>
/var/folders/kq/pm2zcyfs5sggbt9sc5jcmrnm0000gn/T/stack20180711-72036-heg8pd.json:9:7

...

Copy link
Contributor

@patrobinson patrobinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we able to test this? Even if we stub out system call, just to make sure future changes to Stack class are caught :)

return
end

Tempfile.open(['stack', '.json']) do |f|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if it's a YAML template?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was under the impression that proposed_stack.template_body will always be json. Is that incorrect?

If it can be yaml, that's supported too, just need to change the suffix.

Copy link
Contributor

@stevehodgkiss stevehodgkiss Jul 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah the template sent to CF is always JSON from StackMaster (even if the original template is YAML). Turns out YAML is sent through as is https://github.com/envato/stack_master/blob/master/lib/stack_master/template_compilers/yaml.rb

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK we support native YAML templates and we submit them as YAML to CloudFormation. I do not know though if template_body will always be json.

Copy link
Contributor

@stevehodgkiss stevehodgkiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this belongs in StackMaster. I imagine it'd be pretty easy to wire up using a compile command, which I realise now we don't have but I think it'd be a useful addition - stack_master compile region stack | cfn-lint.

return
end

Tempfile.open(['stack', '.json']) do |f|
Copy link
Contributor

@stevehodgkiss stevehodgkiss Jul 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah the template sent to CF is always JSON from StackMaster (even if the original template is YAML). Turns out YAML is sent through as is https://github.com/envato/stack_master/blob/master/lib/stack_master/template_compilers/yaml.rb

end

Tempfile.open(['stack', '.json']) do |f|
f.write(proposed_stack.template_body)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative to using files could be to send the template as STDIN to cfn-lint (assuming it supports that).

@viraptor
Copy link
Contributor Author

I'd be happy with the compile command instead. cfn-lint doesn't do stdin, but you can always use /dev/stdin instead.

I'd prefer the lint one instead for community reasons: It's not a well known project, but has a lot of potential. If it was explicitly raised as a possibility from the help screen, I think that would improve life for a lot of people. (if I didn't have to wait for AWS to break after submitting the stack, I'd save so much time...)

But I can go with consensus. Are the strong feelings about the approach either way?

@stevehodgkiss
Copy link
Contributor

I'm not strongly opposed to including a lint command. Happy for it to be included if it would be useful. I think a compile command would be a good addition regardless of this choice though.

@simpsora
Copy link
Contributor

👍 to compile.
👍 to linting. Given that a good portion of SM's functionality is transforming templates from one format to the other (and hence you're often not working in the native format), it'd be great to have an offline way to ensure the generated template is valid.

@viraptor viraptor changed the title Add a lint command Add lint and compile commands Jul 18, 2018
def perform
unless cfn_lint_available
puts "Failed to run cfn-lint, do you have it installed and available in $PATH?"
return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should exit with a non-zero exit code.

f.write(proposed_stack.template_body)
f.flush
system('cfn-lint', f.path).nil?
puts "cfn-lint run complete"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we pass the exit code from cfn-lint back?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cf-lint doesn't care :( it's always 0, whether it fails or not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Boo

Tempfile.open(['stack', ".#{proposed_stack.template_format}"]) do |f|
f.write(proposed_stack.template_body)
f.flush
system('cfn-lint', f.path).nil?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the point of the nil check here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy-pasta

@viraptor viraptor merged commit c8cfae7 into master Jul 19, 2018
@viraptor viraptor deleted the lint-command branch July 19, 2018 06:16
patrobinson pushed a commit that referenced this pull request Jul 23, 2018
- Adds ACM Cert resolver #227
- Adds compile and lint commands #245
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants