Add pre-commit (#40)

.github/workflows/gh-pages.yml

@@ -1,4 +1,4 @@
-name: Publish to GitHub Pages
+name: gh-pages
 
 on:
  push:
@@ -9,9 +9,9 @@ jobs:
  build:
  runs-on: ubuntu-latest
  steps:
- - uses: actions/checkout@v3
+ - uses: actions/checkout@v4
  - name: Set up Python
- uses: actions/setup-python@v4
+ uses: actions/setup-python@v5
  with:
  python-version: "3.10"
  cache: "pip"

.gitignore

@@ -91,4 +91,5 @@ _build/
 # python-dotenv
 .env
 
-/data
+# Project-specific
+data/

.pre-commit-config.yaml

@@ -0,0 +1,26 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+repos:
+ - repo: https://github.com/pre-commit/pre-commit-hooks
+ rev: v3.2.0
+ hooks:
+ - id: trailing-whitespace
+ - id: end-of-file-fixer
+ - id: check-ast
+ - id: check-yaml
+ - id: check-added-large-files
+ - id: detect-aws-credentials
+ args: [--allow-missing-credentials]
+ - id: detect-private-key
+ - repo: https://github.com/igorshubovych/markdownlint-cli
+ rev: v0.39.0
+ hooks:
+ - id: markdownlint
+ - repo: https://github.com/astral-sh/ruff-pre-commit
+ rev: v0.2.1
+ hooks:
+ - id: ruff
+ types_or: [python, pyi, jupyter]
+ args: [--fix]
+ - id: ruff-format
+ types_or: [python, pyi, jupyter]

LICENSE

@@ -370,4 +370,4 @@ Exhibit B - "Incompatible With Secondary Licenses" Notice
 ---------------------------------------------------------
 
  This Source Code Form is "Incompatible With Secondary Licenses", as
- defined by the Mozilla Public License, v. 2.0.
+ defined by the Mozilla Public License, v. 2.0.

_static/custom.css

@@ -45,4 +45,4 @@ a.current.reference.internal {
 
 #pst-back-to-top {
  background-color: #3eacad;
-}
+}

docs/CONTRIBUTING.md

@@ -63,7 +63,7 @@ To run the notebooks locally, after (1) and (2) above, please follow these steps
  ```
 
  This command will create a new environment based on the specifications provided in the `environment.yml` file.
- 
+
 - [Activate the environment](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#activating-an-environment), run [JupyterLab](https://jupyterlab.readthedocs.io) and execute `notebooks`:
 
  ```shell

docs/bibliography.bib

@@ -24,4 +24,4 @@ @article{BITTNER201734
 author = {Christian Bittner},
 keywords = {OpenStreetMap, Israel, Palestine, Critical cartography, Web 2.0 cartographies, Volunteered geographic information},
 abstract = {In Israel and Palestine, map-making practices were always entangled with contradictive spatial identities and imbalanced power resources. Although an Israeli narrative has largely dominated the ‘cartographic battlefield’, the latest chapter of this story has not been written yet: collaborative forms of web 2.0 cartographies have restructured power relations in mapping practices and challenged traditional monopolies on map and spatial data production. Thus, we can expect web 2.0 cartographies to be a ‘game changer’ for cartography in Palestine and Israel. In this paper, I review this assumption with the popular example of OpenStreetMap (OSM). Following a mixed methods approach, I comparatively analyze the genesis of OSM in Israel and Palestine. Although nationalist motives do not play a significant role on either side, it turns out that the project is dominated by Israeli and international mappers, whereas Palestinians have hardly contributed to OSM. As a result, social fragmentations and imbalances between Israel and Palestine are largely reproduced through OSM data. Discussing the low involvement of Palestinians, I argue that OSM's ground truth paradigm might be a watershed for participation. Presumably, the project's data are less meaningful in some local contexts than in others. Moreover, the seemingly apolitical approach to map only ‘facts on the ground’ reaffirms present spatio-social order and thus the power relations behind it. Within a Palestinian narrative, however, many aspects of the factual material space might appear not as neutral physical objects but as results of suppression, in which case, any ‘accurate’ spatial representation, such as OSM, becomes objectionable.}
-}
+}

docs/requirements.txt

@@ -1,2 +1,2 @@
 docutils==0.17.1
-jupyter-book==1.0.0
+jupyter-book==1.0.0

notebooks/damage-assessment/README.md

@@ -33,12 +33,12 @@ We have employed all the Copernicus [Sentinel-1](https://sentinels.copernicus.eu
 ```{figure} ../../docs/images/damage-assessment-calendar.png
 ---
 ---
-Calendar with Copernicus Sentinel-1 acquisition dates after the war started on past 7th October 2023 until 9th January 2023. 
+Calendar with Copernicus Sentinel-1 acquisition dates after the war started on past 7th October 2023 until 9th January 2023.
 ```
 
 ### 2. Change detection based on time series statistics
 
-For the time series change detection, we have computed all the data pre-war September 2022 until end September 2023 to compute the statistics in non-war situation, and to use those statistics to classify the newer data acquired during the war period October 2023 until the present time with pixels for which had been detected a change (potentially attributable to war damage) using anomaly detection method with different thresholds (i.e. 3 sigma rule and 2.5 sigma rule). 
+For the time series change detection, we have computed all the data pre-war September 2022 until end September 2023 to compute the statistics in non-war situation, and to use those statistics to classify the newer data acquired during the war period October 2023 until the present time with pixels for which had been detected a change (potentially attributable to war damage) using anomaly detection method with different thresholds (i.e. 3 sigma rule and 2.5 sigma rule).
 
 The 3-sigma rule is more conservative and provides more conservative results with false alarms regarding the change detections. 3-sigma rule considers as anomaly values that are lower than the average minus 3 times their standard deviation which it means that are lower than the 99.6% of the value’s normal distribution (measured in non-war conditions) or are included in the 0.15% of possible values, and hence, detected as anomalous. Similarly happens for the 2.5 sigma rule, which pixels are considered as anomaly the ones being the 0.65%. This 2.5 sigma rule may increase some more false alarms, while the 3 sigma rule is considered more conservative anomaly detection rule. See example of this empirical rule below.
 
@@ -70,7 +70,7 @@ For the final assessment of infrastructural damage, in roads, points of interest
 ## Limitations
 
 ```{important}
-The following damage assessment maps are based on preliminary estimates and have not been verified through field survey or satellite imagery. They do not reflect the final estimations by the [World Bank](https://www.worldbank.org/en/country/westbankandgaza). The project team is currently working on procuring high resolution imagery to verify whether the buildings identified as damaged have collapsed or experienced impact. 
+The following damage assessment maps are based on preliminary estimates and have not been verified through field survey or satellite imagery. They do not reflect the final estimations by the [World Bank](https://www.worldbank.org/en/country/westbankandgaza). The project team is currently working on procuring high resolution imagery to verify whether the buildings identified as damaged have collapsed or experienced impact.
 ```
 
 ```{caution}
@@ -83,13 +83,13 @@ Using OpenStreetMap (OSM) and Interferometric Synthetic Aperture Radar (InSAR) f
 - **Temporal Limitations:**
  - **Data Timeliness:** OSM data might not be up to date due to infrequent updates or changes in the landscape that haven't been reflected yet.
  - **InSAR Timing:** InSAR data might not capture the most recent changes or damages, especially in rapidly evolving situations where damage continues to occur after the data collection period.
- 
+
 - **Contextual Understanding:**
  - **Local Knowledge:** OSM data might lack contextual information crucial for assessing damages accurately, such as the original state of buildings or variations in construction materials.
  - **Verification Challenges:** Verifying damages solely based on remote sensing data might lack the on-ground verification necessary for a comprehensive understanding of the situation.
 
 - **Conflict Data**
-ACLED is a crowdsourced dataset and is higly likely that the numbers are underreported. ACLED keep changing their data based on local validation. 
+ACLED is a crowdsourced dataset and is higly likely that the numbers are underreported. ACLED keep changing their data based on local validation.
 ```
 
 ## References

notebooks/nighttime-lights/README.md

@@ -42,5 +42,3 @@ To address these assumptions and limitations, it is crucial to complement nightt
 :filter: docname in docnames
 :style: plain
 ```
-
-

notebooks/nighttime-lights/nighttime-lights-in-jordan.ipynb

@@ -40,11 +40,9 @@
  "import matplotlib.pyplot as plt\n",
  "import pandas as pd\n",
  "from blackmarble.bm_extract import bm_extract\n",
- "from blackmarble.bm_raster import bm_raster\n",
- "from bokeh.models import ColumnDataSource, HoverTool, Legend, Range1d, Span, Text, Title\n",
+ "from bokeh.models import HoverTool, Legend, Span, Title\n",
  "from bokeh.plotting import figure, output_notebook, show\n",
  "from gadm import GADMDownloader\n",
- "from matplotlib.font_manager import get_font_names\n",
  "\n",
  "%load_ext autoreload\n",
  "%autoreload 2"
@@ -1131,7 +1129,9 @@
  ],
  "source": [
  "JO_2 = (\n",
- " VNP46A2.pivot_table(index=\"date\", columns=[\"NAME_2\"], values=[VAR]).resample(\"W\").mean()\n",
+ " VNP46A2.pivot_table(index=\"date\", columns=[\"NAME_2\"], values=[VAR])\n",
+ " .resample(\"W\")\n",
+ " .mean()\n",
  ")\n",
  "\n",
  "JO_1 = (\n",
@@ -1560,7 +1560,7 @@
  ")\n",
  "p.add_layout(\n",
  " Title(\n",
- " text=f\"Weekly Adm2 Radiance Average\",\n",
+ " text=\"Weekly Adm2 Radiance Average\",\n",
  " text_font_size=\"12pt\",\n",
  " text_font_style=\"italic\",\n",
  " ),\n",
@@ -2043,7 +2043,7 @@
  ")\n",
  "p.add_layout(\n",
  " Title(\n",
- " text=f\"Monthly Adm2 Radiance (VNP46A2 and VNP46A3)\",\n",
+ " text=\"Monthly Adm2 Radiance (VNP46A2 and VNP46A3)\",\n",
  " text_font_size=\"12pt\",\n",
  " text_font_style=\"italic\",\n",
  " ),\n",
@@ -2095,7 +2095,7 @@
  " )\n",
  " r.visible = False\n",
  " renderers.append(r)\n",
- " \n",
+ "\n",
  "renderers[-1].visible = True\n",
  "\n",
  "p.legend.location = \"bottom_left\"\n",
@@ -3027,7 +3027,7 @@
  "p.xaxis.major_label_orientation = math.pi / 4\n",
  "p.add_layout(\n",
  " Title(\n",
- " text=f\"Percent change (compared to 2022) in NTL radiance for each second-level administrative division\",\n",
+ " text=\"Percent change (compared to 2022) in NTL radiance for each second-level administrative division\",\n",
  " text_font_size=\"12pt\",\n",
  " text_font_style=\"italic\",\n",
  " ),\n",
@@ -3202,7 +3202,7 @@
  "p.xaxis.major_label_orientation = math.pi / 4\n",
  "p.add_layout(\n",
  " Title(\n",
- " text=f\"Percent change week over week in NTL radiance for each second-level administrative division\",\n",
+ " text=\"Percent change week over week in NTL radiance for each second-level administrative division\",\n",
  " text_font_size=\"12pt\",\n",
  " text_font_style=\"italic\",\n",
  " ),\n",
@@ -3334,7 +3334,7 @@
  "sns.set_style(\"whitegrid\")\n",
  "\n",
  "fig, ax = plt.subplots(figsize=(10, 8))\n",
- "fig.suptitle(f\"Jordan: Weekly Nighttime Lights (2022-2023)\", y=0.95)\n",
+ "fig.suptitle(\"Jordan: Weekly Nighttime Lights (2022-2023)\", y=0.95)\n",
  "\n",
  "JO_0.plot(ax=ax, marker=\"8\")\n",
  "ax.set_title(\"Weekly National Radiance Average\")\n",

notebooks/nighttime-lights/nighttime-lights.ipynb

@@ -39,19 +39,11 @@
  "import geopandas\n",
  "import pandas as pd\n",
  "from blackmarble.extract import bm_extract\n",
- "from blackmarble.raster import bm_raster\n",
  "from bokeh.models import (\n",
- " BasicTicker,\n",
- " ColumnDataSource,\n",
- " Div,\n",
  " HoverTool,\n",
  " Legend,\n",
- " PrintfTickFormatter,\n",
  " Range1d,\n",
  " Span,\n",
- " TabPanel,\n",
- " Tabs,\n",
- " Text,\n",
  " Title,\n",
  ")\n",
  "from bokeh.plotting import figure, output_notebook, show"
@@ -12059,7 +12051,7 @@
  ")\n",
  "p.add_layout(\n",
  " Title(\n",
- " text=f\"Monthly NTL Radiance Average (VNP46A2 and VNP46A3) for each second-level administrative division\",\n",
+ " text=\"Monthly NTL Radiance Average (VNP46A2 and VNP46A3) for each second-level administrative division\",\n",
  " text_font_size=\"12pt\",\n",
  " text_font_style=\"italic\",\n",
  " ),\n",
@@ -12663,7 +12655,7 @@
  "\n",
  " p.add_layout(\n",
  " Title(\n",
- " text=f\"Weekly percent change (2022 baseline) in NTL radiance for each second-level administrative division\",\n",
+ " text=\"Weekly percent change (2022 baseline) in NTL radiance for each second-level administrative division\",\n",
  " text_font_size=\"12pt\",\n",
  " text_font_style=\"italic\",\n",
  " ),\n",
@@ -12875,7 +12867,7 @@
  " )\n",
  " p.add_layout(\n",
  " Title(\n",
- " text=f\"Weekly percent change (2022 baseline) in NTL radiance\",\n",
+ " text=\"Weekly percent change (2022 baseline) in NTL radiance\",\n",
  " text_font_size=\"12pt\",\n",
  " text_font_style=\"italic\",\n",
  " ),\n",
@@ -13158,7 +13150,7 @@
  ")\n",
  "p.add_layout(\n",
  " Title(\n",
- " text=f\"Weekly percent change week over week in NTL radiance for each second-level administrative division\",\n",
+ " text=\"Weekly percent change week over week in NTL radiance for each second-level administrative division\",\n",
  " text_font_size=\"12pt\",\n",
  " text_font_style=\"italic\",\n",
  " ),\n",

notebooks/sample-indicators/health.ipynb

@@ -19,11 +19,9 @@
  "source": [
  "import geopandas\n",
  "import matplotlib.pyplot as plt\n",
- "import numpy as np\n",
  "import pandas as pd\n",
  "from matplotlib.ticker import FuncFormatter\n",
- "from palettable.cartocolors.qualitative import Prism_10, Vivid_10\n",
- "from palettable.tableau import ColorBlind_10, TableauMedium_10\n",
+ "from palettable.tableau import TableauMedium_10\n",
  "\n",
  "%config InlineBackend.figure_formats = ['svg']\n",
  "%matplotlib inline"
@@ -2033,10 +2031,12 @@
  ],
  "source": [
  "fig, ax = plt.subplots(figsize=(10, 6))\n",
- "plt.suptitle(\"Gaza: Percentage of Damaged Healthcare Locations\", y=1.01, fontsize=20, x=0.47)\n",
+ "plt.suptitle(\n",
+ " \"Gaza: Percentage of Damaged Healthcare Locations\", y=1.01, fontsize=20, x=0.47\n",
+ ")\n",
  "\n",
- "df.pivot_table(\"perc\", \"ADM2_EN\", \"type\")[['Health']].plot(\n",
- " ax=ax, kind=\"barh\", color=TableauMedium_10.mpl_colors, legend = False\n",
+ "df.pivot_table(\"perc\", \"ADM2_EN\", \"type\")[[\"Health\"]].plot(\n",
+ " ax=ax, kind=\"barh\", color=TableauMedium_10.mpl_colors, legend=False\n",
  ")\n",
  "\n",
  "# Add labels and customization\n",
@@ -2051,11 +2051,12 @@
  "ax.yaxis.set_ticks_position(\"left\")\n",
  "ax.xaxis.set_ticks_position(\"bottom\")\n",
  "ax.grid(which=\"both\", linestyle=\"--\", linewidth=0.25, color=\"gray\", alpha=0.5)\n",
- "#ax.legend(bbox_to_anchor=(1.01, 0.9))\n",
+ "# ax.legend(bbox_to_anchor=(1.01, 0.9))\n",
  "\n",
  "ax.set_title(\n",
  " \"Estimated percentage of damaged healthcare locations for each second-level administrative division\",\n",
- " fontsize=14,loc='left'\n",
+ " fontsize=14,\n",
+ " loc=\"left\",\n",
  ")\n",
  "ax.text(\n",
  " 0,\n",