delay_status_endpoint.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyOtn52/XuiQR5ULmkzd88ce"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "code",
      "source": [
        "# HTTP \n",
        "import requests\n",
        "\n",
        "# XML \n",
        "from bs4 import BeautifulSoup"
      ],
      "metadata": {
        "id": "rhUr8uzxJ82-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Airport Delay Status Endpoint\n",
        "The Air Traffic Control System Command Center [FAQ](https://www.fly.faa.gov/FAQ/faq.jsp), at the bottom under \"Developer's Questions\" contains the following:\n",
        "\n",
        "\n",
        "\"I am building a web site, and would like access to your delay data. Is there some way for me to receive the data from you?\n",
        "\n",
        "- Our airport status information is available in XML format. [Here](https://www.fly.faa.gov/AirportStatus.dtd) is the DTD. [Here](https://www.fly.faa.gov/flyfaa/xmlAirportStatus.jsp) is the status information.\"\n",
        "\n",
        "This JSP file ends up redirecting to https://nasstatus.faa.gov/api/airport-status-information which holds the XML content. We can parse this content according to the DTD (document type definition) file. As an example, here is the content of airport-status-information at the time of writing:\n",
        "\n",
        "\n",
        "```\n",
        "<AIRPORT_STATUS_INFORMATION>\n",
        "  <Update_Time>Tue Oct 18 03:10:16 2022 GMT</Update_Time>\n",
        "  <Dtd_File>https://www.fly.faa.gov/AirportStatus.dtd</Dtd_File>\n",
        "  <Delay_type>\n",
        "    <Name>General Arrival/Departure Delay Info</Name>\n",
        "    <Arrival_Departure_Delay_List>\n",
        "      <Delay>\n",
        "        <ARPT>DFW</ARPT>\n",
        "        <Reason>TM INITIATIVES:NON-RNAV</Reason>\n",
        "        <Arrival_Departure Type=\"Departure\">\n",
        "          <Min>15 minutes</Min>\n",
        "          <Max>29 minutes</Max>\n",
        "          <Trend>Decreasing</Trend>\n",
        "        </Arrival_Departure>\n",
        "      </Delay>\n",
        "    </Arrival_Departure_Delay_List>\n",
        "  </Delay_type>\n",
        "</AIRPORT_STATUS_INFORMATION>\n",
        "```\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "e3m_GKHnC_8y"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "This first batch of **parse_** methods use the DTD information to parse the different types of delay lists we can encounter. The relevent lines of the DTD file are included as a comment at the top of each method. \n",
        "TODO: Parse DTD on the fly instead of hardcoding."
      ],
      "metadata": {
        "id": "H2J7yRDjq5yK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "<!ELEMENT CTOP_List                    (CTOP+)>\n",
        "<!ELEMENT CTOP                         (Program_Name, Reason, CTOP_Start_Time, CTOP_End_Time, Avg, FCA_Data)>\n",
        "<!ELEMENT FCA_Data                     (FCA+)>\n",
        "<!ELEMENT FCA                          (FCA_Name, FCA_Start_DateTime, FCA_End_DateTime, (Line|Polygon|Circle), Floor, Ceiling)>\n",
        "Also see: https://www.fly.faa.gov/What_s_New/XML_is_changing.jsp \n",
        "\"\"\"\n",
        "def parse_ctop_list(ctop_list_tag):\n",
        "\n",
        "  master_ctop_list = []\n",
        "\n",
        "  # TODO\n",
        "  # Loop through CTOP elements\n",
        "\n",
        "  # Loop through children FCA elements\n",
        "\n",
        "  return master_ctop_list"
      ],
      "metadata": {
        "id": "qJwIilvlqaj-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "<!ELEMENT Airport_Closure_List         (Airport*)>\n",
        "<!ELEMENT Airport                      (ARPT, Reason, Start, Reopen)>\n",
        "\"\"\"\n",
        "def parse_closure_list(closure_list_tag):\n",
        "  \n",
        "  master_closure_list = []\n",
        "\n",
        "  # Loop through the Airport elements\n",
        "  airport_tags = closure_list_tag.find_all(\"airport\")\n",
        "  for cur_airport in airport_tags:\n",
        "\n",
        "    # Grab the 4 attributes: airport name, closure reason, start and reopen datetimes\n",
        "    cur_closure_dict = {\n",
        "        \"arpt\" : \"\",\n",
        "        \"reason\" : \"\",\n",
        "        \"start\" : \"\",\n",
        "        \"reopen\" : \"\"\n",
        "    }\n",
        "\n",
        "    cur_closure_dict[\"arpt\"] = cur_airport.find(\"arpt\").text.strip()\n",
        "    cur_closure_dict[\"reason\"] = cur_airport.find(\"reason\").text.strip()\n",
        "    cur_closure_dict[\"start\"] = cur_airport.find(\"start\").text.strip()\n",
        "    cur_closure_dict[\"reopen\"] = cur_airport.find(\"reopen\").text.strip()\n",
        "\n",
        "    # Append to master\n",
        "    master_closure_list.append(cur_closure_dict)\n",
        "  \n",
        "  return master_closure_list"
      ],
      "metadata": {
        "id": "G-rD2IQJRG_k"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "<!ELEMENT Ground_Stop_List             (Program*)>\n",
        "<!ELEMENT Program                      (ARPT, Reason, End_Time)>\n",
        "\"\"\"\n",
        "def parse_ground_stop_list(groundstop_list_tag):\n",
        "\n",
        "  master_stop_list = []\n",
        "\n",
        "  # Loop through Program elements\n",
        "  program_tags = groundstop_list_tag.find_all(\"program\")\n",
        "  for stop_program in program_tags:\n",
        "\n",
        "    # Grab airport name, reason, and end time\n",
        "    cur_stop_dict = {\n",
        "        \"arpt\" : \"\",\n",
        "        \"reason\" : \"\",\n",
        "        \"endtime\" : \"\"\n",
        "    }\n",
        "\n",
        "    cur_stop_dict[\"arpt\"] = stop_program.find(\"arpt\").text.strip()\n",
        "    cur_stop_dict[\"reason\"] = stop_program.find(\"reason\").text.strip()\n",
        "    cur_stop_dict[\"endtime\"] = stop_program.find(\"end_time\").text.strip()\n",
        "\n",
        "    # Append to master\n",
        "    master_stop_list.append(cur_stop_dict)\n",
        "\n",
        "  return master_stop_list"
      ],
      "metadata": {
        "id": "hNqiAh2lROZS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "<!ELEMENT Ground_Delay_List            (Ground_Delay*)>\n",
        "<!ELEMENT Ground_Delay                 (ARPT, Reason, Avg, Max)>\n",
        "\"\"\"\n",
        "def parse_ground_delay_list(grounddelay_list_tag):\n",
        "\n",
        "  master_grounddelay_list = []\n",
        "\n",
        "  # Loop through Ground_Delay elements\n",
        "  ground_delay_tags = grounddelay_list_tag.find_all(\"ground_delay\")\n",
        "  for ground_delay in ground_delay_tags:\n",
        "\n",
        "    # Get airport name, reason, average delay, max delay\n",
        "    cur_ground_delay = {\n",
        "        \"arpt\" : \"\",\n",
        "        \"reason\" : \"\",\n",
        "        \"avg\" : \"\",\n",
        "        \"max\" : \"\"\n",
        "    }\n",
        "\n",
        "    cur_ground_delay[\"arpt\"] = ground_delay.find(\"arpt\").text.strip()\n",
        "    cur_ground_delay[\"reason\"] = ground_delay.find(\"reason\").text.strip()\n",
        "    cur_ground_delay[\"avg\"] = ground_delay.find(\"avg\").text.strip()\n",
        "    cur_ground_delay[\"max\"] = ground_delay.find(\"max\").text.strip()\n",
        "\n",
        "    # Append\n",
        "    master_grounddelay_list.append(cur_ground_delay)\n",
        "\n",
        "  return master_grounddelay_list"
      ],
      "metadata": {
        "id": "7_oTDHnNRYxG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "<!ELEMENT Airspace_Flow_List           (Airspace_Flow*)>\n",
        "<!ELEMENT Airspace_Flow                (CTL_Element, Reason, FCA_Start_DateTime, FCA_End_DateTime, AFP_StartTime, AFP_EndTime, (Line|Polygon|Circle), Avg, Floor, Ceiling)>\n",
        "\n",
        "<!ELEMENT Line                         (Point,Point+)>\n",
        "<!ELEMENT Polygon                      (PointsList)>\n",
        "<!ELEMENT Circle                       (Center)>\n",
        "<!ELEMENT PointsList                   (Point,Point+)>\n",
        "<!ELEMENT Center                       EMPTY>\n",
        "<!ELEMENT Point                        EMPTY>\n",
        "<!ATTLIST Point Lat                    CDATA #REQUIRED>\n",
        "<!ATTLIST Point Long                   CDATA #REQUIRED>\n",
        "<!ATTLIST Center Lat                   CDATA #REQUIRED>\n",
        "<!ATTLIST Center Long                  CDATA #REQUIRED>\n",
        "<!ATTLIST Circle Radius                CDATA #REQUIRED>\n",
        "\"\"\"\n",
        "def parse_flow_list(flow_list_tag):\n",
        "  # TODO\n",
        "  return []"
      ],
      "metadata": {
        "id": "TOgJVF9qRcQy"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "<!ELEMENT Arrival_Departure_Delay_List (Delay*)>\n",
        "<!ELEMENT Delay                        (ARPT, Reason, Arrival_Departure, Arrival_Departure?)>\n",
        "\n",
        "<!ELEMENT Arrival_Departure            (Min, Max, Trend)>\n",
        "<!ATTLIST Arrival_Departure Type       (Arrival|Departure) #REQUIRED>\n",
        "\"\"\"\n",
        "def parse_ad_delay_list(ad_list_tag):\n",
        "\n",
        "  master_list = []\n",
        "  \n",
        "  # Loop through the Delay elements\n",
        "  delay_tags = ad_list_tag.find_all(\"delay\")\n",
        "  for cur_delay in delay_tags:\n",
        "\n",
        "    # As in the DTD excerpt and XML example above, there is a separate substructure for delays to arrivals and delays to departure (whichever exist(s))\n",
        "    # We will extract the information (Min, Max, and Trend) from each into a subdictionary\n",
        "    cur_ad_dict = {\n",
        "        \"arpt\" : \"\",\n",
        "        \"reason\" : \"\",\n",
        "        \"arrival_delay\" : {},\n",
        "        \"departure_delay\" : {}\n",
        "\n",
        "    }\n",
        "\n",
        "    # Grab the airport name and delay reason\n",
        "    cur_ad_dict[\"arpt\"] = cur_delay.find(\"arpt\").text.strip()\n",
        "    cur_ad_dict[\"reason\"] = cur_delay.find(\"reason\").text.strip()\n",
        "\n",
        "    # If they exist, will be one structure for arrival delays and one for departure. Type attribute holds either \"Arrival\" or \"Departure\"\n",
        "    arrivals_and_departs = cur_delay.find_all(\"arrival_departure\")\n",
        "    for arrive_or_depart in arrivals_and_departs:\n",
        "\n",
        "      if arrive_or_depart.get(\"type\").strip().lower() == \"arrival\":\n",
        "        cur_ad_dict[\"arrival_delay\"][\"min\"] = arrive_or_depart.find(\"min\").text.strip()\n",
        "        cur_ad_dict[\"arrival_delay\"][\"max\"] = arrive_or_depart.find(\"max\").text.strip()\n",
        "        cur_ad_dict[\"arrival_delay\"][\"trend\"] = arrive_or_depart.find(\"trend\").text.strip()\n",
        "\n",
        "      elif arrive_or_depart.get(\"type\").strip().lower() == \"departure\":\n",
        "        cur_ad_dict[\"departure_delay\"][\"min\"] = arrive_or_depart.find(\"min\").text.strip()\n",
        "        cur_ad_dict[\"departure_delay\"][\"max\"] = arrive_or_depart.find(\"max\").text.strip()\n",
        "        cur_ad_dict[\"departure_delay\"][\"trend\"] = arrive_or_depart.find(\"trend\").text.strip()\n",
        "\n",
        "    # Append current delay to master list\n",
        "    master_list.append(cur_ad_dict)\n",
        "\n",
        "  return master_list"
      ],
      "metadata": {
        "id": "KQiAcXxARiE7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The above methods all support **get_airport_status_info**, the method which actually calls the endpoint:"
      ],
      "metadata": {
        "id": "tAj6Gcf3s0Q2"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dlw4DYftCXl_"
      },
      "outputs": [],
      "source": [
        "# Main method which calls /api/airport-status-information\n",
        "def get_airport_status_info():\n",
        "\n",
        "  master_delay_dict = {\n",
        "      \"update_time\" : \"\",\n",
        "      \"delays_by_type\" : {}\n",
        "  }\n",
        "\n",
        "  # Get XML content\n",
        "  req_headers = { \"User-Agent\" : \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36\" }\n",
        "  response = requests.get(url = \"https://nasstatus.faa.gov/api/airport-status-information\", headers = req_headers)\n",
        "  response.raise_for_status()\n",
        "\n",
        "  xml_content = response.content\n",
        "  soup = BeautifulSoup(xml_content, \"lxml\")\n",
        "\n",
        "  # First grab the update time\n",
        "  try:\n",
        "    master_delay_dict[\"update_time\"] = soup.airport_status_information.update_time.text.strip()\n",
        "  except:\n",
        "    print(\"Error reading update time from AIRPORT_STATUS_INFORMATION. Check XML structure for changes\")\n",
        "    return\n",
        "\n",
        "  # Loop through Delay_type elements. As per the DTD, the following delay_types are possible: \n",
        "  # CTOP_List, Airport_Closure_List, Ground_Stop_List, Ground_Delay_List, Airspace_Flow_List, Arrival_Departure_Delay_List\n",
        "  delay_types = soup.find_all(\"delay_type\")\n",
        "  for cur_delay_type in delay_types:\n",
        "\n",
        "    \"\"\"\n",
        "    We expect 2 children: a \"name\" element and then one of the above listed types of elements. The name element is just a description of the type of delay list and is disregarded\n",
        "    Determine which type of list we are in. We rely on this being the next sibling of the name element\n",
        "    We also rely on there only being one list of each type, or the last (of each type) will be the only one recorded due to the dict structure of master_delay_dict[\"delays_by_type\"]. May optimize\n",
        "    \"\"\"\n",
        "    cur_delay_type_list = []\n",
        "    list_name = \"\"\n",
        "    list_type = \"\"\n",
        "    list_name = cur_delay_type.find(\"name\").text.strip()\n",
        "    list_type = cur_delay_type.find(\"name\").next_sibling.name.strip()\n",
        "\n",
        "    # Parse the list with the correct helper method\n",
        "    if list_type == \"ctop_list\":\n",
        "      cur_delay_type_list = parse_ctop_list(cur_delay_type)\n",
        "    elif list_type ==  \"airport_closure_list\":\n",
        "      cur_delay_type_list = parse_closure_list(cur_delay_type)\n",
        "    elif list_type ==  \"ground_stop_list\":\n",
        "      cur_delay_type_list = parse_ground_stop_list(cur_delay_type)\n",
        "    elif list_type ==  \"ground_delay_list\":\n",
        "      cur_delay_type_list = parse_ground_delay_list(cur_delay_type)\n",
        "    elif list_type ==  \"airspace_flow_list\":\n",
        "      cur_delay_type_list = parse_flow_list(cur_delay_type)\n",
        "    elif list_type ==  \"arrival_departure_delay_list\":\n",
        "      cur_delay_type_list = parse_ad_delay_list(cur_delay_type)\n",
        "    else:\n",
        "      print(\"Encountered an unknown delay list type: {}. Will be appended with no data. Check XML structure for changes\".format(list_type))\n",
        "\n",
        "    # Add an entry to master_delay_dict[\"delays_by_type\"] for each type of delay list we get. The list type is the key name and the list of delays is the value. Again, this will result in only the last list \n",
        "    # of each type being saved. May update to support multiple lists of the same type although I have not seen the API response structured in such a way. \n",
        "    master_delay_dict[\"delays_by_type\"][list_type] = cur_delay_type_list\n",
        "\n",
        "  return master_delay_dict\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(get_airport_status_info())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-A27RaOmM7Wc",
        "outputId": "4190ab02-1079-41f9-fb1a-c17c09ccfbc9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'update_time': 'Tue Oct 18 17:47:35 2022 GMT', 'delays_by_type': {'arrival_departure_delay_list': [{'arpt': 'LAS', 'reason': 'RWY:Construction', 'arrival_delay': {}, 'departure_delay': {'min': '16 minutes', 'max': '30 minutes', 'trend': 'Increasing'}}], 'airport_closure_list': [{'arpt': 'SBA', 'reason': '!SBA 10/062 SBA AD AP CLSD DLY 0700-1200 2210180700-2210211200', 'start': 'Oct 18 at 07:00 UTC.', 'reopen': 'Oct 21 at 12:00 UTC.'}]}}\n"
          ]
        }
      ]
    }
  ]
}