-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reuse image for functions with same build metadata #2466
Comments
One workaround I have found is to use a different AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Parameters:
Tag:
Type: String
Default: latest
Description: Docker tag to build and deploy.
Globals:
Function:
Timeout: 5
Resources:
FooFunction:
Type: AWS::Serverless::Function
Properties:
PackageType: Image
ImageConfig:
Command:
- foo
Metadata:
DockerTag: !Sub foo-${Tag}
DockerContext: .
BarFunction:
Type: AWS::Serverless::Function
Properties:
PackageType: Image
ImageConfig:
Command:
- bar
Metadata:
DockerTag: !Sub bar-${Tag}
DockerContext: . |
Came here to report the same thing. The problem comes from here: https://github.com/aws/aws-sam-cli/blob/develop/samcli/lib/build/app_builder.py#L183 The built_artifacts only lists the first function because that's the only one that actually performed any build work. I'm still tracing through the code to figure out how that gets populated, but thank you for the workaround. That will at least get me past my current roadblock. |
I accidentally opened #2514 without realizing it duplicated this issue. Closing that one and joining the conversation here. I was using a symlink as a workaround in my post. But I can confirm that I can remove |
FYI, I found another bug, though I believe this one is much deeper than SAM (or at least the part that runs locally). It appears that if you specify a Command in the template it will lose track of the EntryPoint. I found that I had to specify both Command and Entrypoint ('/lambda-entrypoint.sh') in order to be able to successfully override the command. If you run into issues about missing entry points then give that a shot. |
@jasonmk That is a know bug with how the CFN resources for Lambda (currently) work. The correct teams are aware internally already. |
This solves the problem of getting the The context loading for each function build causes it to take a while on larger containers even if the build itself is fast. Is it possible to only build once? |
@nitsujri I ran into the exact same problem, as you say, even though docker eventually knows to use its cache, it still takes a while for it to load. I'm not sure what contributes to this loading time but I'm assuming it's docker itself rather than aws sam cli. If that's the case, I haven't myself been able to find anything within docker to speed this up. Taking a look at sam, this line seems to show it explicitly decides to not use the same caching logic as non image builds
This makes sense as, even though the context is set for each function, not everything within that context may be copied into the image as that is controlled within the Dockerfile and/or dockerignore files. However maybe at least caching based on the context is a step in the right direction - might still cover most user cases... Am happy to contribute if there is an acceptable path forward to utilize sam cache build strategy for images! |
@billyshambrook yeah the time I'm complaining about is docker build copying the context. THe time is consumed for each function in the current workaround, in my case 3x. For now, since SAM must build all functions individually, I'm avoiding using echo "== Docker Build..."
docker build --tag $IMAGE_URI \
--file ./docker/Dockerfile.production $LAMBY_BUILD_CACHE_DIR
echo "== ECR Login..."
aws ecr get-login-password --profile $AWS_PROFILE | docker login --username AWS --password-stdin $IMAGE_REPOSITORY
echo "== ECR Push..."
docker push $IMAGE_URI
echo "== SAM deploy..."
sam deploy \
--profile "${AWS_PROFILE}" \
--template-file ./template.yaml \
--stack-name "lamby-discovery-${RAILS_ENV}" \
--image-repository "$IMAGE_REPOSITORY" \
--capabilities "CAPABILITY_IAM" \
--parameter-overrides \
RailsEnv="${RAILS_ENV}" \
RailsImageUri=$IMAGE_URI This completely sidesteps the problem in a rather ugly way. This does mean I lost the Going back to the issue at hand. It would probably have to be similar to:
I can agree though that feels quite brittle and the number of corner cases could be quite high. Perhaps manual control of the |
I'm hitting this too, in testing out SAM. I'm concerned, because I expect to have dozens of lambdas that reuse the same image (same .net code base). Waiting while uploading what amounts to the same image dozens of times to different ecr repos for every build doesn't sound great. Perhaps the concerns about automatic detection of the image being identical can be addressed by allowing us to pass a variable or flag on 2nd, 3rd function resources' ImageURI:. Or maybe that's already a possibility, if the format of the auto-created image name were constant/predictable (automatic repo names currently contain guids or hashes), and things were guaranteed to happen in order. |
Commenting for support, I would also like to be able to reuse the same image for multiple lambdas. |
Commenting for support also, this would be very good |
Commenting for support. The above workaround doesn't seem to work anymore. |
@inhumantsar Hi may I please ask why the workaround doesn't work for you? Thanks! |
commenting for support. i too would like to use the same image for multiple lambdas. |
commenting for support. i too would like to use the same image for multiple lambdas. |
Description:
It looks like if you have 2 functions using
PackageType: Image
using the same Dockerfile,sam build
does not correctly addImageUri
to all functions.I have traced through the code that all functions are added to the same
build_definition
but the build results are not copied to all functions. There is an explicit if condition to only do this toZIP
, is there a reason for this or it's just a feature not supported yet?aws-sam-cli/samcli/lib/build/build_strategy.py
Line 117 in 12efe15
Steps to reproduce:
Example template:
Observed result:
The "built" template only has
ImageUri
on the BarFunction:Expected result:
Expect to re-use the same image for multiple functions.
I can however see an issue because the image name includes the first function name.
Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
sam --version
: 1.13.2The text was updated successfully, but these errors were encountered: