Setup pre-commit for GEF (#984)

This PR adds `pre-commit` to facilitate and unify the formatting of
everything in this project (code, test, docs, ci).

Aside from `.pre-commit-config.yml` and `docs/.markdownlint.yaml`, all
the other changes are purely cosmetic, applying markdownlint and fixing
here and there to make it pass.

It also creates a new CI job that'll simply execute `pre-commit` for new
PRs.

.github/CONTRIBUTING.md

@@ -1,22 +1,27 @@
-## Contributing to GEF ##
+## Contributing to GEF
 
-### License ###
+## License
 
-`gef` is placed under [MIT license](https://github.com/hugsy/gef/blob/main/LICENSE) which provides Open-Source access to the code and its use.
+`gef` is placed under [MIT license](https://github.com/hugsy/gef/blob/main/LICENSE) which provides
+Open-Source access to the code and its use.
 
-By contributing to `gef` code through the _Pull Requests_ mechanism, you accept to release the code written by you under the said license.
+By contributing to `gef` code through the _Pull Requests_ mechanism, you accept to release the code
+written by you under the said license.
 
+## Submitting a Patch
 
-### Submitting a Patch ###
-
-1. Fork `gef` repository (requires GitHub account). Sending a patch from the
+1. Fork `gef` repository (requires GitHub account). Sending a patch from the
  `patch` or `git diff --patch` commands is not accepted.
-1. Write the changes in your local repo.
- * Respect the coding style (same indentation format, explicit names as possible)
- * Comment your code
- * If you add a new feature/GDB command, also write the adequate
- documentation (in [`docs/`](docs/))
-1. Submit a pull request, make sure it is made against the `main` branch
-1. The contributors will review your patch. If it is approved, the change will
+1. All the packages required for testing and documenting are listed in `tests/requirements.txt`
+1. Adjust your development environment to GEF's: this is achieved using
+ [`pre-commit`](https://pre-commit.com/), and getting setup is simply done by
+1. Installing `pre-commit` PIP package (part of the `requirements.txt` file)
+1. Setup `pre-commit` : `pre-commit install`
+1. Write the changes in your local repo making sure to respect the coding style (same indentation
+ format, explicit names as possible), comment your code sufficiently so it becomes maintainable by
+ someone other than you. Finally if you add a new feature/GDB command, also write the adequate
+ documentation (in [`docs/`](docs/))
+1. Submit a pull request
+1. The contributors will review your patch. If it is approved, the change will
  be merged via the GitHub, and you will be seen as contributors. If it needs
  additional work, the repo owner will respond with useful comments.

.github/FUNDING.yml

@@ -1,4 +1,3 @@
 # These are supported funding model platforms
 
 github: [hugsy,]
-

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -7,7 +7,7 @@ body:
  id: md_welcome
  attributes:
  value: |
- Complete this form is for reporting bugs in GEF. Incomplete/invalid report will simply being ignored or closed. 
+ Complete this form is for reporting bugs in GEF. Incomplete/invalid report will simply being ignored or closed.
  Also note that this is not the place to ask installation or usage problem. Use the Discord channel for that.
 - type: textarea
  attributes:
@@ -38,9 +38,9 @@ body:
  multiple: false
  options:
  - "No"
- - "Yes" 
+ - "Yes"
  validations:
- required: true 
+ required: true
 - type: checkboxes
  attributes:
  label: Architecture impacted

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,31 +1,20 @@
-
-## Description/Motivation/Screenshots
+## Description
 
 <!-- Describe technically what your patch does. -->
-<!-- Why is this change required? What problem does it solve? -->
-<!-- Why is this patch will make a better world? -->
-<!-- How does this look? Add a screenshot if you can -->
 
+<!-- Why is this change required? What problem does it solve? -->
 
-## Against which architecture was this tested ?
-
-"Tested" indicates that the PR works *and* the unit test (see `docs/testing.md`) run passes without issue.
+<!-- Why is this patch will make a better world? -->
 
- * [ ] x86-32
- * [ ] x86-64
- * [ ] ARM
- * [ ] AARCH64
- * [ ] MIPS
- * [ ] POWERPC
- * [ ] SPARC
- * [ ] RISC-V
+<!-- How does this look? Add a screenshot if you can -->
 
+<!-- Annotate your PR with label proper labels (architecture impacted, type of improvement, etc.)
 
 ## Checklist
 
 <!-- N.B.: Your patch won't be reviewed unless fulfilling the following base requirements: -->
 <!--- Put an `x` in all the boxes that are complete, or that don't apply -->
-- [ ] My code follows the code style of this project.
-- [ ] My change includes a change to the documentation, if required.
-- [ ] If my change adds new code, [adequate tests](docs/testing.md) have been added.
-- [ ] I have read and agree to the **CONTRIBUTING** document.
+-  [ ] My code follows the code style of this project.
+-  [ ] My change includes a change to the documentation, if required.
+-  [ ] If my change adds new code, [adequate tests](docs/testing.md) have been added.
+-  [ ] I have read and agree to the **CONTRIBUTING** document.

.github/stale.yml

@@ -20,6 +20,6 @@ closeComment: >
  complete description; or create a new issue.
  Thank you for your contributions.
 # Set to true to ignore issues in a milestone (defaults to false)
-exemptMilestones: true 
+exemptMilestones: true
 # Set to true to ignore issues with an assignee (defaults to false)
 exemptAssignees: true

.github/workflows/coverage.yml

@@ -2,13 +2,14 @@ name: CI Coverage for PR
 
 on:
  pull_request:
- types: [opened, edited]
 
 jobs:
  coverage:
  runs-on: ubuntu-latest
  steps:
  - uses: actions/checkout@v3
+ with:
+ fetch-depth: '0'
 
  - name: Run test coverage
  id: get_coverage
@@ -28,10 +29,14 @@ jobs:
  new_score=$(cat docs/coverage/gef_py.html | grep pc_cov | sed 's?.*<span class="pc_cov">\([^%]*\)%</span>?\1?g')
  diff_score=$(python -c "print(${new_score} - ${current_score})")
  commit=$(git rev-parse HEAD)
+ include_tests=$(git diff ${{ github.event.before }} ${{ github.event.after }} --compact-summary | egrep --count '^ tests/' || true)
+ include_docs=$(git diff ${{ github.event.before }} ${{ github.event.after }} --compact-summary | egrep --count '^ docs/' || true)
  echo "commit=${commit}" >> $GITHUB_OUTPUT
  echo "new_coverage_score=${new_score}" >> $GITHUB_OUTPUT
  echo "current_coverage_score=${current_score}" >> $GITHUB_OUTPUT
  echo "diff_score=${diff_score}" >> $GITHUB_OUTPUT
+ echo "include_tests=${include_tests}" >> $GITHUB_OUTPUT
+ echo "include_docs=${include_docs}" >> $GITHUB_OUTPUT
 
  - uses: actions/github-script@v6
  env:
@@ -41,12 +46,54 @@ jobs:
  SCORE_DIFF: ${{ steps.get_coverage.outputs.diff_score }}
  with:
  script: |
+ const tests_changes = ${{ steps.get_coverage.outputs.include_tests }};
+ const docs_changes = ${{ steps.get_coverage.outputs.include_docs }};
  const comment = `## 🤖 Coverage Update
 
  * Commit: ${process.env.COMMIT}
  * Current Coverage: ${process.env.SCORE_OLD}%
  * New Coverage: ${process.env.SCORE_NEW}%
  * Diff: ${process.env.SCORE_DIFF}
+
+ To this point, this PR:
+ * ${(tests_changes == 0) ? "**does not** include" : "includes"} changes to tests
+ * ${(docs_changes == 0) ? "**does not** include" : "includes"} changes to documentation
+
  `;
+
  const { owner, repo, number } = context.issue;
  await github.rest.issues.createComment({ owner, repo, issue_number: number, body: comment });
+
+ try {
+ if(docs_changes > 0) {
+ await github.rest.issues.addLabels({
+ owner: owner,
+ repo: repo,
+ issue_number: number,
+ labels: ['documentation']
+ });
+ } else {
+ await github.rest.issues.removeLabel({
+ owner: owner,
+ repo: repo,
+ issue_number: number,
+ name: 'documentation'
+ });
+ }
+
+ if(tests_changes > 0) {
+ await github.rest.issues.addLabels({
+ owner: owner,
+ repo: repo,
+ issue_number: number,
+ labels: ['automation/ci']
+ });
+ } else {
+ await github.rest.issues.removeLabel({
+ owner: owner,
+ repo: repo,
+ issue_number: number,
+ name: 'automation/ci'
+ });
+ }
+ } catch (err) {}

.github/workflows/run-tests.yml

@@ -67,8 +67,3 @@ jobs:
  run: |
  make -C tests/binaries -j ${{ env.GEF_CI_NB_CPU }}
  python${{ env.PY_VER }} -m pytest --forked -n ${{ env.GEF_CI_NB_CPU }} -v -k "not benchmark" tests/
-
- - name: Run linter
- run: |
- python${{ env.PY_VER }} -m pylint --rcfile=$(pwd)/.pylintrc gef.py tests/*/*.py
-

.github/workflows/validate.yml

@@ -0,0 +1,31 @@
+name: Validation
+
+on:
+ pull_request:
+ branches:
+ - main
+
+jobs:
+ pre_commit:
+ name: Check formatting
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - uses: actions/setup-python@v3
+ - uses: pre-commit/[email protected]
+
+ docs_link_check:
+ name: Check URLs in docs
+ runs-on: ubuntu-latest
+ permissions:
+ contents: read
+ steps:
+ - name: checkout
+ uses: actions/checkout@v2
+ - name: Check links
+ uses: lycheeverse/[email protected]
+ env:
+ GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
+ with:
+ args: --exclude-mail --accept=401 --no-progress 'docs/**/*.md'
+ fail: false

.pre-commit-config.yaml

@@ -0,0 +1,25 @@
+default_stages: [pre-commit, pre-push]
+fail_fast: false
+repos:
+ - repo: https://github.com/pre-commit/pre-commit-hooks
+ rev: v4.4.0
+ hooks:
+ - id: end-of-file-fixer
+ - id: trailing-whitespace
+
+ - repo: https://github.com/pycqa/pylint
+ rev: v3.0.0a6
+ hooks:
+ - id: pylint
+
+ - repo: https://github.com/igorshubovych/markdownlint-cli
+ rev: v0.35.0
+ hooks:
+ - id: markdownlint-docker
+ args:
+ - --config=docs/.markdownlint.yaml
+ - --ignore=docs/api/gef.md
+ - --ignore=docs/obsolete/docs/index.md
+ - --ignore=docs/index.md
+ - --ignore=README.md
+ - "docs/**/*.md"

README.md

@@ -15,12 +15,11 @@ development. Application developers will also benefit from it, as GEF lifts a gr
 GDB obscurity, avoiding repeating traditional commands, or bringing out the relevant information
 from the debugging runtime.
 
-## Instant Setup ##
+## Instant Setup
 
 Simply make sure you have [GDB 8.0 or higher](https://www.gnu.org/s/gdb) compiled with Python3.6+
 bindings, then:
 
-
 ```bash
 # via the install script
 ## using curl
@@ -44,18 +43,18 @@ You can immediately see that GEF is correctly installed by launching GDB:
 
 A few of `GEF` features include:
 
-* **One** single GDB script
-* Entirely **architecture agnostic**, **NO** dependencies: `GEF` is battery-included and [is
+*  **One** single GDB script
+*  Entirely **architecture agnostic**, **NO** dependencies: `GEF` is battery-included and [is
  installable instantly](https://hugsy.github.io/gef/#setup)
-* **Fast** limiting the number of dependencies and optimizing code to make the commands as fast as
+*  **Fast** limiting the number of dependencies and optimizing code to make the commands as fast as
  possible
-* Provides a great variety of commands to drastically change your experience in GDB.
-* [**Easily** extensible](https://hugsy.github.io/gef/api/) to create other commands by providing
+*  Provides a great variety of commands to drastically change your experience in GDB.
+*  [**Easily** extensible](https://hugsy.github.io/gef/api/) to create other commands by providing
 d more comprehensible layout to GDB Python API.
-* Full Python3 support ([Python2 support was
+*  Full Python3 support ([Python2 support was
  dropped](https://github.com/hugsy/gef/releases/tag/2020.03) - see
  [`gef-legacy`](https://github.com/hugsy/gef-legacy)).
-* Built around an architecture abstraction layer, so all commands work in any GDB-supported
+*  Built around an architecture abstraction layer, so all commands work in any GDB-supported
  architecture such as x86-32/64, ARMv5/6/7, AARCH64, SPARC, MIPS, PowerPC, etc.
 * Suited for real-life apps debugging, exploit development, just as much as CTF
 * And a lot more commands contributed by the community available on
@@ -64,34 +63,30 @@ d more comprehensible layout to GDB Python API.
 Check out the [Screenshot page](docs/screenshots.md) for more or [try it
 online](https://demo.gef.blah.cat) (user:`gef`/password:`gef-demo`)
 
-
-## Documentation ##
+## Documentation
 
 Unlike other GDB plugins, GEF has an extensive and up-to-date
 [documentation](https://hugsy.github.io/gef/). Users are recommended to refer to it as it may help
 them in their attempts to use GEF. In particular, new users should navigate through it (see the
 [FAQ](https://hugsy.github.io/gef/faq/) for common installation problems), and the problem persists,
 try to reach out for help on the Discord channel or submit an issue.
 
-
-## Current status ##
+## Current status
 
 | Documentation |License | Compatibility | CI Tests (`main`) |
 |:---:|:---:|:---|--|
 | [![Documentation](https://github.com/hugsy/gef/actions/workflows/generate-docs.yml/badge.svg)](https://github.com/hugsy/gef/actions/workflows/generate-docs.yml) | [![MIT](https://img.shields.io/packagist/l/doctrine/orm.svg?maxAge=2592000?style=plastic)](https://github.com/hugsy/gef/blob/main/LICENSE) | [![Python 3](https://img.shields.io/badge/Python-3-green.svg)](https://github.com/hugsy/gef/) | [![CI Test for GEF](https://github.com/hugsy/gef/actions/workflows/run-tests.yml/badge.svg)](https://github.com/hugsy/gef/actions/workflows/run-tests.yml) |
 
-
-## Contribute ##
+## Contribute
 
 To get involved, refer to the [Contribution
 documentation](https://hugsy.github.io/gef/#contribution) and the
 [guidelines](https://github.com/hugsy/gef/blob/main/.github/CONTRIBUTING.md) to start.
 
-## Sponsors ##
+## Sponsors
 
 Another way to contribute to keeping the project alive is by sponsoring it! Check out [the
 sponsoring documentation](https://hugsy.github.io/gef/#sponsors) for details so you can be part of
 the list of those [awesome sponsors](https://github.com/sponsors/hugsy).
 
-
-## Happy Hacking 🍻 ##
+## Happy Hacking 🍻

docs/.markdownlint.yaml

@@ -1,3 +1,6 @@
+#
+# Rules: https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md
+#
 # Default state for all rules
 default: true
 
@@ -10,7 +13,7 @@ MD001: true
 # MD002/first-heading-h1/first-header-h1 - First heading should be a top-level heading
 MD002:
  # Heading level
- level: 1
+ level: 2
 
 # MD003/heading-style/header-style - Heading style
 MD003:
@@ -161,7 +164,7 @@ MD032: true
 # MD033/no-inline-html - Inline HTML
 MD033:
  # Allowed elements
- allowed_elements: []
+ allowed_elements: ["img"]
 
 # MD034/no-bare-urls - Bare URL used
 MD034: true
@@ -195,21 +198,15 @@ MD040:
 # MD041/first-line-heading/first-line-h1 - First line in a file should be a top-level heading
 MD041:
  # Heading level
- level: 1
+ level: 2
  # RegExp for matching title in front matter
  front_matter_title: "^\\s*title\\s*[:=]"
 
 # MD042/no-empty-links - No empty links
 MD042: true
 
 # MD043/required-headings/required-headers - Required heading structure
-MD043:
- # List of headings
- headings: []
- # List of headings
- headers: []
- # Match case of headings
- match_case: false
+MD043: false
 
 # MD044/proper-names - Proper names should have the correct capitalization
 MD044: