Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Deps + Clean up Vendor File #1197

Merged
merged 3 commits into from
May 10, 2024
Merged

Fix Deps + Clean up Vendor File #1197

merged 3 commits into from
May 10, 2024

Conversation

sb-benohe
Copy link
Contributor

@sb-benohe sb-benohe commented May 10, 2024

JIRA Ticket:
N/A

User Story or Bug Summary:

dependancy for prod build fail hash check

What Does This PR Do?

fixes hases and cleans up reqs versions

What Should Reviewers Watch For?

That this PR builds locally

What Security Implications Does This PR Have?

Submitters should complete the following questionnaire:

  • If the answer to any of the questions below is Yes, then here's a link to the associated Security Impact Assessment (SIA), security checklist, or other similar document in Confluence: N/A.
    • Does this PR add any new software dependencies? Yes or No.
    • Does this PR modify or invalidate any of our security controls? Yes or No.
    • Does this PR store or transmit data that was not stored or transmitted before? Yes or No.
  • If the answer to any of the questions below is Yes, then please add StewGoin as a reviewer, and note that this PR should not be merged unless/until he also approves it.
    • Do you think this PR requires additional review of its security implications for other reasons? Yes or No.

What Needs to Be Merged and Deployed Before this PR?

N/A

Any Migrations?

  • Yes, there are migrations
    • The migrations should be run PRIOR to the code being deployed
    • The migrations should be run AFTER the code is deployed
    • There is a more complicated migration plan (downtime, etc)
  • No migrations

Submitter Checklist

I have gone through and verified that...:

  • This PR is reasonably limited in scope, to help ensure that:
    1. It doesn't unnecessarily tie a bunch of disparate features, fixes, refactorings, etc. together.
    2. There isn't too much of a burden on reviewers.
    3. Any problems it causes have a small "blast radius".
    4. It'll be easier to rollback if that becomes necessary.
  • I have named this PR and its branch such that they'll be automatically be linked to the (most) relevant Jira issue, per: https://confluence.atlassian.com/adminjiracloud/integrating-with-development-tools-776636216.html.
  • This PR includes any required documentation changes, including README updates and changelog / release notes entries.
  • All new and modified code is appropriately commented, such that the what and why of its design would be reasonably clear to engineers, preferably ones unfamiliar with the project.
  • All tech debt and/or shortcomings introduced by this PR are detailed in TODO and/or FIXME comments, which include a JIRA ticket ID for any items that require urgent attention.
  • Reviews are requested from both:
    • At least two other engineers on this project, at least one of whom is a senior engineer or owns the relevant component(s) here.
    • Any relevant engineers on other projects (e.g. BFD, SLS, etc.).
  • Any deviations from the other policies in the DASG Engineering Standards are specifically called out in this PR, above.
    • Please review the standards every few months to ensure you're familiar with them.

@sb-benohe sb-benohe self-assigned this May 10, 2024
Copy link
Contributor

@loganbertram loganbertram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know why all the hash changes seemingly without any change in expected version? Everything looks good and this for sure addresses the last error, but I'd be curious what went wrong for our convo on Monday.

Comment on lines +243 to +244
--hash=sha256:6e6ff3db2d8dd0c986b4eec8554c8e4f919b5c1ff62a5b4390c17aff2ed6e5c4 \
--hash=sha256:ddc24a0a8280a0430baa37aff11f28574720af05888c62b7cfe71d219f4599d3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the big one I was looking for. This definitely resolves the error in the last build.

Copy link
Contributor

@jimmyfagan jimmyfagan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks my build locally, working on summarizing the errors and identifying a fix to the Dockerfile to workaround it, but wanted to submit a review to make sure we don't deliver before then. Should have another comment in a few minutes!

@jimmyfagan
Copy link
Contributor

I got an error on my docker-compose:

0.631 ERROR: Could not find a version that satisfies the requirement cffi==1.15.1 (from versions: none)
0.631 ERROR: No matching distribution found for cffi==1.15.1

But adding RUN pip install cffi==1.15.1 resolved that.. only to get

1.559 ERROR: Could not find a version that satisfies the requirement markupsafe==2.1.5 (from versions: none)
1.559 ERROR: No matching distribution found for markupsafe==2.1.5

Of course adding RUN pip install markupsafe==2.1.5 to the Dockerfile fixed that as well. All in all, here is the new Dockerfile that worked for me:

FROM python:3.8
ENV PYTHONUNBUFFERED 1
RUN useradd -m -s /bin/bash DEV
USER DEV
ADD . /code
WORKDIR /code
RUN python -m venv /tmp/venv
RUN . /tmp/venv/bin/activate
ENV PATH="/tmp/venv/bin:${PATH}"
RUN pip install --upgrade pip
RUN pip config set global.trusted-host "pypi.org files.pythonhosted.org pypi.python.org"
RUN pip install --upgrade pip-tools
RUN pip install --upgrade setuptools --trusted-host pypi.python.org --trusted-host=files.pythonhosted.org
RUN pip install backports.zoneinfo
RUN pip install cffi==1.15.1
RUN pip install charset-normalizer==3.1.0
RUN pip install cryptography==42.0.7
RUN pip install debugpy==1.6.7 --trusted-host pypi.python.org --trusted-host=files.pythonhosted.org
RUN pip install markupsafe==2.1.5
RUN pip install newrelic==8.8.0
RUN pip install pillow==10.3.0
RUN pip install psycopg2-binary==2.9.6
RUN pip install pyyaml==6.0.1
RUN pip install wrapt==1.15.0
RUN pip install -r requirements/requirements.dev.txt --no-index --find-links ./vendor/

Hopefully we can just add this for now, but yea add that to the list of things we would ideally not have to do during dependency updates.

@jimmyfagan jimmyfagan self-requested a review May 10, 2024 14:19
@sb-benohe
Copy link
Contributor Author

@jimmyfagan That might be because that image might be ARM_X64 Vs AMD_X64 so the packages might be different

Copy link
Contributor

@jimmyfagan jimmyfagan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made some edits to the Dockerfile and docker-compose.yml to get this to work for Mac, looking good now!

@sb-benohe sb-benohe enabled auto-merge (squash) May 10, 2024 18:17
@sb-benohe sb-benohe merged commit 00f20f0 into master May 10, 2024
6 checks passed
@jimmyfagan jimmyfagan deleted the reqs_fix branch May 10, 2024 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants