Code Coverage with pytest and Github Actions - Qodana does not show code coverage

[Darnol]

Hi there, I'm new to using Qodana and run into a problem with my setup where I use Github Actions and want to test some Python funcitons with Pytest.
I posted the same question to Stackoverflow [here](Code Coverage with pytest and Github Actions - Qodana does not show code coverage), with no success so far.

I tried to set up everything according to
https://www.jetbrains.com/help/qodana/code-coverage.html

The only difference: I'm using pytest instead of unittest, as suggested by the GitHub Actions template. Apart from this I', trying to follow the documentation.

https://github.com/qodana/qodana-coverage/blob/7360c03be1f44a4ed0e591218977005b07dd569e/.github/workflows/python.yml

Below you can find my qodana.yaml and my GitHub Actions yaml. The Action does run, the tests are executed (GitHub Action Screenshot) and an artifact with the report in XML format is uploaded. Problem: Qodana does not seem to use the report at all (see screenshot), the Code Coverage in the top right is still set to "Not Set".

My Question: Does anybody know how I have to configure Qodana in order to display Code Coverage results? The documentation says the default profile should be fine. I have not set any other Qodana profiles, just using the default.

---

qodana.yaml

version: "1.0"
linter: jetbrains/qodana-python-community:2023.3
exclude:
  - name: PyShadowingNamesInspection
  - name: PyListCreationInspection
  - name: PyPep8Inspection
  - name: PyPep8NamingInspection
  - name: PyUnusedLocalInspection
  - name: PyMethodMayBeStaticInspection
  - name: PyIncorrectDocstringInspection
  - name: All
    paths:
      - path/to/excluded/folder

---

GitHub Action yaml

name: Qodana
on:
  workflow_dispatch:
  pull_request:
  push:
    branches: # Specify your branches here
      - main # The 'main' branch
      - 'releases/*' # The release branches

jobs:
  qodana:
    runs-on: ubuntu-latest

    permissions:
      contents: write
      pull-requests: write
      checks: write
    
    steps:
      - name: Checkout Code
        uses: actions/checkout@v3
        with:
          ref: ${{ github.event.pull_request.head.sha }}  # to check out the actual pull request commit, not the merge commit
          fetch-depth: 0  # a full history is required for pull request analysis
      
      - name: Set up Python 3.10
        uses: actions/setup-python@v3
        with:
          python-version: "3.10"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
    
      - name: Run tests with coverage
        run: |
          coverage run -m pytest -s
          coverage xml -o .qodana/code-coverage/coverage.xml

      - name: Archive coverage data
        uses: actions/upload-artifact@v2
        with:
          name: py-coverage-data
          path: .qodana/code-coverage

      - name: 'Qodana Scan'
        uses: JetBrains/[email protected]
        env:
          QODANA_TOKEN: ${{ secrets.QODANA_TOKEN }}
        with:
          args: --baseline,qodana.sarif.json

I tried to follow the Qodana documentation to set up code coverage for Python. I expected the code coverage results to show up in the report overview of the Qodana result.

[brichbash]

Hello,

The code coverage feature is available only in paid linters. As I can see, you use jetbrains/qodana-python-community:2023.3. Could you tryjetbrains/qodana-python:2023.3 instead? We provide a free 60-day trial for the Ultimate plans.

[brichbash]

Could you add to your GitHub Qodana job:

upload-result: true

Run your Qodana job again and then share the generated artifact? You can send it to [email protected] if you don't want to share it publicly

[Darnol]

EDIT: Somehow when I send a mail to [email protected] they come back as "undeliverable". Anyways, I just posted the artifacts here for you to check them. Thank you so much! There is the log of the last run of the GitHub Action, the uploaded pytest report and the artifcat generated (qodana-report.zip). I hope this helps with the troubleshooting!

logs_22890838179.zip
py-coverage-data.zip
qodana-report.zip

Thanks for your suggestion. Just to make sure, my GitHub Action yaml looks now like this.
I only added the line upload-result: true in the step Qodana Scan with uses: JetBrains/[email protected]

The yaml now looks like this, with the final line upload-result: true added. Is this correct?

name: Qodana
on:
  workflow_dispatch:
  pull_request:
  push:
    branches: # Specify your branches here
      - main # The 'main' branch
      - 'releases/*' # The release branches

jobs:
  qodana:
    runs-on: ubuntu-latest

    permissions:
      contents: write
      pull-requests: write
      checks: write
    
    steps:
      - name: Checkout Code
        uses: actions/checkout@v3
        with:
          ref: ${{ github.event.pull_request.head.sha }}  # to check out the actual pull request commit, not the merge commit
          fetch-depth: 0  # a full history is required for pull request analysis
      
      - name: Set up Python 3.10
        uses: actions/setup-python@v3
        with:
          python-version: "3.10"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
    
      - name: Run tests with coverage
        run: |
          coverage run -m pytest -s
          coverage xml -o .qodana/code-coverage/coverage.xml

      - name: Archive coverage data
        uses: actions/upload-artifact@v2
        with:
          name: py-coverage-data
          path: .qodana/code-coverage

      - name: 'Qodana Scan'
        uses: JetBrains/[email protected]
        env:
          QODANA_TOKEN: ${{ secrets.QODANA_TOKEN }}
        with:
          args: --baseline,qodana.sarif.json
          upload-result: true

[brichbash]

Thank you!
The configuration is correct. Although some files from qodana-report.zip are empty (zero bytes) which is quite unusual. Are you sure the archive was downloaded correctly?

[Darnol]

Hi there, I just tried again, I download the zip from the github actions summary. I'm pretty sure that download is correct. Or is there any other way to download the generated qodana-report file?

[Darnol]

Hi, are there any news to this issue?