refactor(core): playback

.github/workflows/docs.yml

@@ -2,10 +2,9 @@ name: build docs
 
 on:
  workflow_dispatch:
- # pull_request_target:
- # push:
- # branches:
- # - master
+ push:
+ branches:
+ - master
 
 permissions:
  contents: write

docs/usage/examples.md

@@ -22,25 +22,27 @@ You can find even more usage examples under [`tests/`](https://github.com/cathay
 
 #### Paginate all pages
 Queries for next page as long as user doesn't enter `x`, or if there are no pages left.
+In each iteration, rows are upserted and saved to the [cache](../usage/cli.md#directories).
 
 Note that pagination cannot be run in parallel: fetching page N requires information from page N-1.
 
 === "Jupyter cell"
 
- ```py
+ ```py hl_lines="5 8"
  --8<-- "docs/usage/scripts/10_flight_list.py:script1"
  ```
 
- 1. First attempt to load existing table from the cache, otherwise it creates an empty in-memory arrow table for us to concat to.
+ 1. First attempt to load existing table from the [cache](../usage/cli.md#directories), otherwise it creates an empty in-memory arrow table for us to concat to.
+ 2. [Upserts the data][fr24.core.FlightListArrow.concat], replacing older records with new ones.
 
 === "`data.df`"
 
  ```
  --8<-- "docs/usage/scripts/10_flight_list.py:df1"
  ```
-<!-- 
+
 ### Playback
-*API reference: [fr24.core.FR24.playback][], [fr24.core.PlaybackAPI.fetch][]*
+*API reference: [fr24.core.PlaybackService][], [fr24.core.PlaybackService.fetch][]*
 
 #### Miracle on the Hudson
 Downloads the flight trajectory for [UA1549](https://en.wikipedia.org/wiki/US_Airways_Flight_1549)
@@ -53,13 +55,13 @@ Downloads the flight trajectory for [UA1549](https://en.wikipedia.org/wiki/US_Ai
 
  1. From https://www.flightradar24.com/data/pinned/ua1549-2fb3041#2fb3041.
 
-=== "`pb.data.df`"
+=== "`data.df`"
 
  ```
  --8<-- "docs/usage/scripts/11_playback.py:df0"
  ```
 
-=== "`pb.data.metadata`"
+=== "`data.metadata`"
 
  ```py
  --8<-- "docs/usage/scripts/11_playback.py:metadata0"
@@ -70,27 +72,25 @@ Saves trajectory data to disk, reads the track and metadata from it.
 
 === "Jupyter cell"
 
- ```py hl_lines="7 9 11 12"
+ ```py hl_lines="8 10"
  --8<-- "docs/usage/scripts/11_playback.py:script1"
  ```
 
- 1. Delete existing parquet files, if it exists.
- 2. Save the parquet to disk.
- 3. Delete the arrow table to make room.
- 4. Load the parquet from disk.
+ 1. Saves the parquet to the [cache](../usage/cli.md#directories).
+ 2. Load the parquet from the [cache](../usage/cli.md#directories).
 
-=== "`pb.data.df`"
+=== "`data_local.df`"
 
  ```
  --8<-- "docs/usage/scripts/11_playback.py:df0"
  ```
 
-=== "`pb.data.metadata`"
+=== "`data_local.metadata`"
 
  ```py
  --8<-- "docs/usage/scripts/11_playback.py:metadata0"
  ```
-### Live Feed
+<!-- ### Live Feed
 *API reference: [fr24.core.FR24.livefeed][], [fr24.core.LiveFeedAPI.fetch][]*
 
 #### Live

docs/usage/quickstart.md

@@ -38,7 +38,7 @@ However, the [FR24][fr24.core.FR24] class provides a convenient wrapper around t
 - Has three services:
  - [**Live Feed**][fr24.core.FR24.livefeed]: snapshot of all aircraft state vectors
  - [**Flight List**][fr24.core.FlightListService]: all historical flights for a given aircraft registration or flight number
- - [**Playback**][fr24.core.FR24.playback]: historical trajectory for one flight.
+ - [**Playback**][fr24.core.PlaybackService]: historical trajectory for one flight.
 
 Each service has its own async `.fetch()` method to retrieve raw data from the API. `.to_arrow()` can then be used to transform to an Apache Arrow table, and used to perform caching and downstream `pandas` operations.
 
@@ -174,13 +174,13 @@ You can always check its exact location using `.data.fp`. In general, where it g
 
 ### Storage Location
 
-- [Flight list][fr24.core.FR24.flight_list]
+- [Live feed][fr24.core.FR24.livefeed]
+ - `feed/{timestamp}.parquet`
+- [Flight list][fr24.core.FlightListService]
  - `flight_list/reg/{reg.upper()}.parquet`, or
  - `flight_list/flight/{iata_flight_num.upper()}.parquet`
-- [Playback][fr24.core.FR24.playback]
+- [Playback][fr24.core.PlaybackService]
  - `playback/{fr24_hex_id.lower()}.parquet`
-- [Live feed][fr24.core.FR24.livefeed]
- - `feed/{timestamp}.parquet`
 
 It should resemble the following on Linux:
 ```
@@ -223,9 +223,9 @@ Each service inherits from the [fr24.base.ServiceBase][] and have similar APIs d
 See the [examples gallery](./examples.md) to learn more.
 
 !!! tip
- Pyarrow unfortunately do not provide type hints. You can however, generate the stubs with:
+ Pyarrow unfortunately do not provide type hints. You can however, generate the stubs to your `site-packages` directory with:
  ```sh
- stubgen -p pyarrow -o $PATH_TO_SITE_PACKAGES
+ $ stubgen -p pyarrow -o $(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
  ```
 
 Intersphinx for this project could be found [here](https://cathaypacific8747.github.io/fr24/objects.inv).

docs/usage/scripts/00_introduction.py

@@ -58,7 +58,7 @@ async def main(): # (1)!
 
 from fr24.core import FR24
 
-client = httpx.AsyncClient(transport=httpx.AsyncHTTPTransport(retries=5))
+client = httpx.AsyncClient(http2=True, transport=httpx.AsyncHTTPTransport(retries=5))
 async with FR24(client) as fr24:
  ...
 # --8<-- [end:client-sharing]

docs/usage/scripts/10_flight_list.py

@@ -38,7 +38,7 @@ async def my_full_list() -> None:
  data = fr24.flight_list.load(reg="B-HPB") # (1)!
  async for response in fr24.flight_list.fetch_all(reg="B-HPB"):
  data_new = response.to_arrow()
- data.concat(data_new, inplace=True)
+ data.concat(data_new, inplace=True) # (2)!
  if input() == "x":
  break
  data.save()

docs/usage/scripts/11_playback.py

@@ -7,11 +7,11 @@
 
 async def my_playback() -> None:
  async with FR24() as fr24:
- pb = fr24.playback(flight_id=0x2FB3041) # (1)!
- pb.data._add_api_response(await pb.api._fetch())
- print(pb.data.df)
- rich.print(pb.data.metadata)
- pb.data._save_parquet()
+ response = await fr24.playback.fetch(0x2FB3041) # (1)!
+ data = response.to_arrow()
+ print(data.df)
+ rich.print(data.metadata)
+ data.save()
 
 await my_playback()
 # --8<-- [end:script0]
@@ -63,15 +63,13 @@ async def my_playback() -> None:
 
 async def my_playback() -> None:
  async with FR24() as fr24:
- fl = fr24.playback(flight_id=0x2FB3041)
- fl.data.fp.unlink(missing_ok=True) # (1)!
- fl.data._add_api_response(await fl.api._fetch())
- fl.data._save_parquet() # (2)! 
-
- fl.data._clear() # (3)!
- fl.data._from_file() # (4)!
- print(fl.data.df)
- rich.print(fl.data.metadata)
+ response = await fr24.playback.fetch(0x2FB3041)
+ data = response.to_arrow()
+ data.save() # (1)!
+ # some time later...
+ data_local = fr24.playback.load(0x2FB3041) # (2)!
+ print(data_local.df)
+ rich.print(data_local.metadata)
 
 await my_playback()
 # --8<-- [end:script1]

src/fr24/base.py

@@ -126,6 +126,9 @@ def df(self) -> pd.DataFrame:
  data.attrs = self.ctx
  return data
 
+ def __repr__(self) -> str:
+ return f"{self.__class__.__name__}(ctx={self.ctx}, data={self.data})"
+
 
 class ServiceBase(ABC):
  """A service to handle the API and disk operations."""