exp/lighthorizon: enforce the limit from request on the response size

[sreuland]

check the page limit parameter on tx and ops listing requests, it should determine the response results size.
Closes #4430

Confirmed by live testing, exported and indexed small ledger range,

START_LEDGER= 1538223
STOP_LEDGER= 1538323

/transactions processes the cursor and limit query string parameters, if no limit defaults to 10

request by cursor: l=1538223, t=1, o=1

GET localhost:8080/transactions?cursor=6606617478959105&limit=2

{
  "_links": {
    "self": {
      "href": "https://localhost:8080/transactions?cursor=\u0026limit=0\u0026order="
    },
    "next": {
      "href": "https://localhost:8080/transactions?cursor=6606621773930497\u0026limit=0\u0026order="
    },
    "prev": {
      "href": "https://localhost:8080/transactions?cursor=6606621773926401\u0026limit=0\u0026order=asc"
    }
  },
  "_embedded": {
    "records": [
      {
        "memo": "xdr.Memo{Type: xdr.MemoTypeMemoNone}",
        "_links": {
          "self": {
            "href": "https://localhost:8080/transactions/544469b76cd90978345a4734a0ce69a9d0ddb4a6595a7afc503225a77826722a"
          },
          "account": {
            "href": ""
          },
          "ledger": {
            "href": ""
          },
          "operations": {
            "href": ""
          },
          "effects": {
            "href": "https://localhost:8080/transactions/544469b76cd90978345a4734a0ce69a9d0ddb4a6595a7afc503225a77826722a/effects"
          },
          "precedes": {
            "href": "https://localhost:8080/effects?order=asc\u0026cursor=6606621773926401"
          },
          "succeeds": {
            "href": "https://localhost:8080/effects?order=desc\u0026cursor=6606621773926401"
          },
          "transaction": {
            "href": ""
          }
        },
        "id": "544469b76cd90978345a4734a0ce69a9d0ddb4a6595a7afc503225a77826722a",
        "paging_token": "6606621773926401",
        "successful": true,
        "hash": "544469b76cd90978345a4734a0ce69a9d0ddb4a6595a7afc503225a77826722a",
        "ledger": 1538224,
        "created_at": "2022-06-17T23:29:42Z",
        "source_account": "GBGTCH47BOEEKLPHHMR2GOK6KQFGL3O7Q53FIZTJ7S7YEDWYJ5IUDJDJ",
        "source_account_sequence": "",
        "fee_account": "",
        "fee_charged": "84000",
        "max_fee": "0",
        "operation_count": 28,
        "envelope_xdr": "AAAAAgAAAABNMR-fC4hFLec7I6M5XlQKZe3fh3ZUZmn8v4IO2E9RQQABSCAADOFXAABwTwAAAAAAAAAAAAAAHAAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-OwAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-PgAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-OAAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-OgAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-PQAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-OQAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-PAAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-QwAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-QQAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-PwAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-QgAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-QAAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-RAAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAAAAAAAAAAAQCYloAAAAAAAAZ-RQAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAEDnlMQh53yD1RDVAAAAAAAAAAAAAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAIHPJwL02EVCvo-8wAAAAAAAAAAAAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAMK2uoSl41QEUaEEAAAAAAAAAAAAAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAQOeVELled8CsaWcAAAAAAAAAAAAAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAUSF58KBZa5CVSZpAAAAAAAAAAAAAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAYVte0C3eGiAqvtQQAAAAAAAAAAAAAAAAAAAADAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAcZVDsOkaLRDZcvfAAAAAAAAAAAAAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAcZVDsZMg_SGDvpLAAAAAAAAAAAAAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAYVte0Gte2_BnYMCQAAAAAAAAAAAAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAUSF58NIsz0DKkMfwAAAAAAAAAAAAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAQOeVEbwopqGsg3aAAAAAAAAAAAAAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAMK2uoe_bRSHe4OVAAAAAAAAAAAAAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAIHPJwaZQbUGYRTZAAAAAAAAAAAAAAAAAAAAAMAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAEDnlMCU7cgAkB-ngAAAAAAAAAAAAAAAAAAAAB2E9RQQAAAEDOHy2IQJTNEz9kgUbHYolo_qjWo5s1ZkHOo70AQM2DRVWSNb4Py043HLbgZ6B5WdgkAs3gzoQ9sYk77KVZ5CsG",
        "result_xdr": "AAAAAAABSCAAAAAAAAAAHAAAAAAAAAADAAAAAAAAAAAAAAACAAAAAAAAAAMAAAAAAAAAAAAAAAIAAAAAAAAAAwAAAAAAAAAAAAAAAgAAAAAAAAADAAAAAAAAAAAAAAACAAAAAAAAAAMAAAAAAAAAAAAAAAIAAAAAAAAAAwAAAAAAAAAAAAAAAgAAAAAAAAADAAAAAAAAAAAAAAACAAAAAAAAAAwAAAAAAAAAAAAAAAIAAAAAAAAADAAAAAAAAAAAAAAAAgAAAAAAAAAMAAAAAAAAAAAAAAACAAAAAAAAAAwAAAAAAAAAAAAAAAIAAAAAAAAADAAAAAAAAAAAAAAAAgAAAAAAAAAMAAAAAAAAAAAAAAACAAAAAAAAAAwAAAAAAAAAAAAAAAIAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAABNMR-fC4hFLec7I6M5XlQKZe3fh3ZUZmn8v4IO2E9RQQAAAAAABn5GAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAEDnlMQh53yD1RDVAAAAAAAAAAAAAAAAAAAAADAAAAAAAAAAAAAAAAAAAAAE0xH58LiEUt5zsjozleVApl7d-HdlRmafy_gg7YT1FBAAAAAAAGfkcAAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAFVU0RWAAAAAC92QZMdIcwR90sUUJm7MSb0tJclDlFBKjJHVG7wE0NNAAAAAAgc8nAvTYRUK-j7zAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAATTEfnwuIRS3nOyOjOV5UCmXt34d2VGZp_L-CDthPUUEAAAAAAAZ-SAAAAAFFVVJWAAAAAC92QZMdIcwR90sUUJm7MSb0tJclDlFBKjJHVG7wE0NNAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAAADCtrqEpeNUBFGhBAAAAAAAAAAAAAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAABNMR-fC4hFLec7I6M5XlQKZe3fh3ZUZmn8v4IO2E9RQQAAAAAABn5JAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAQOeVELled8CsaWcAAAAAAAAAAAAAAAAAAAAADAAAAAAAAAAAAAAAAAAAAAE0xH58LiEUt5zsjozleVApl7d-HdlRmafy_gg7YT1FBAAAAAAAGfkoAAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAFVU0RWAAAAAC92QZMdIcwR90sUUJm7MSb0tJclDlFBKjJHVG7wE0NNAAAAABRIXnwoFlrkJVJmkAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAATTEfnwuIRS3nOyOjOV5UCmXt34d2VGZp_L-CDthPUUEAAAAAAAZ-SwAAAAFFVVJWAAAAAC92QZMdIcwR90sUUJm7MSb0tJclDlFBKjJHVG7wE0NNAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAAAGFbXtAt3hogKr7UEAAAAAAAAAAAAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAABNMR-fC4hFLec7I6M5XlQKZe3fh3ZUZmn8v4IO2E9RQQAAAAAABn5MAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAcZVDsOkaLRDZcvfAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAE0xH58LiEUt5zsjozleVApl7d-HdlRmafy_gg7YT1FBAAAAAAAGfk0AAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAFFVVJWAAAAAC92QZMdIcwR90sUUJm7MSb0tJclDlFBKjJHVG7wE0NNAAAAAB2FvShg76SwZMg_SAAAAAAAAAAAAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAAATTEfnwuIRS3nOyOjOV5UCmXt34d2VGZp_L-CDthPUUEAAAAAAAZ-TgAAAAFVU0RWAAAAAC92QZMdIcwR90sUUJm7MSb0tJclDlFBKjJHVG7wE0NNAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAAAGUd8mxnYMCQa17b8AAAAAAAAAAAAAAAAAAAADAAAAAAAAAAAAAAAAAAAAABNMR-fC4hFLec7I6M5XlQKZe3fh3ZUZmn8v4IO2E9RQQAAAAAABn5PAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAVC2xdMqQx_DSLM9AAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAE0xH58LiEUt5zsjozleVApl7d-HdlRmafy_gg7YT1FBAAAAAAAGflAAAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAFFVVJWAAAAAC92QZMdIcwR90sUUJm7MSb0tJclDlFBKjJHVG7wE0NNAAAAABDRjn1rIN2gbwopqAAAAAAAAAAAAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAAATTEfnwuIRS3nOyOjOV5UCmXt34d2VGZp_L-CDthPUUEAAAAAAAZ-UQAAAAFVU0RWAAAAAC92QZMdIcwR90sUUJm7MSb0tJclDlFBKjJHVG7wE0NNAAAAAUVVUlYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAAADJnfsXe4OVB79tFIAAAAAAAAAAAAAAAAAAAADAAAAAAAAAAAAAAAAAAAAABNMR-fC4hFLec7I6M5XlQKZe3fh3ZUZmn8v4IO2E9RQQAAAAAABn5SAAAAAVVTRFYAAAAAL3ZBkx0hzBH3SxRQmbsxJvS0lyUOUUEqMkdUbvATQ00AAAABRVVSVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAAIZGP3ZhFNkGmUG1AAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAE0xH58LiEUt5zsjozleVApl7d-HdlRmafy_gg7YT1FBAAAAAAAGflMAAAABVVNEVgAAAAAvdkGTHSHMEfdLFFCZuzEm9LSXJQ5RQSoyR1Ru8BNDTQAAAAFFVVJWAAAAAC92QZMdIcwR90sUUJm7MSb0tJclDlFBKjJHVG7wE0NNAAAAAAQxGOcJAfp4CU7cgAAAAAAAAAAAAAAAAA==",
        "result_meta_xdr": "",
        "fee_meta_xdr": "",
        "memo_type": "MemoTypeMemoNone",
        "signatures": [
          "2E9RQQAAAEDOHy2IQJTNEz9kgUbHYolo_qjWo5s1ZkHOo70AQM2DRVWSNb4Py043HLbgZ6B5WdgkAs3gzoQ9sYk77KVZ5CsG"
        ]
      },
      {
        "memo": "xdr.MemoText(\"psp:1405\")",
        "_links": {
          "self": {
            "href": "https://localhost:8080/transactions/5fef21d5ef75ecf18d65a160cfab17dca8dbf6dbc4e2fd66a510719ad8dddb09"
          },
          "account": {
            "href": ""
          },
          "ledger": {
            "href": ""
          },
          "operations": {
            "href": ""
          },
          "effects": {
            "href": "https://localhost:8080/transactions/5fef21d5ef75ecf18d65a160cfab17dca8dbf6dbc4e2fd66a510719ad8dddb09/effects"
          },
          "precedes": {
            "href": "https://localhost:8080/effects?order=asc\u0026cursor=6606621773930497"
          },
          "succeeds": {
            "href": "https://localhost:8080/effects?order=desc\u0026cursor=6606621773930497"
          },
          "transaction": {
            "href": ""
          }
        },
        "id": "5fef21d5ef75ecf18d65a160cfab17dca8dbf6dbc4e2fd66a510719ad8dddb09",
        "paging_token": "6606621773930497",
        "successful": false,
        "hash": "5fef21d5ef75ecf18d65a160cfab17dca8dbf6dbc4e2fd66a510719ad8dddb09",
        "ledger": 1538224,
        "created_at": "2022-06-17T23:29:42Z",
        "source_account": "GCFJN22UG6IZHXKDVAJWAVEQ3NERGCRCURR2FHARNRBNLYFEQZGML4PW",
        "source_account_sequence": "",
        "fee_account": "",
        "fee_charged": "3000",
        "max_fee": "0",
        "operation_count": 1,
        "envelope_xdr": "AAAAAgAAAACKlutUN5GT3UOoE2BUkNtJEwoipGOinBFsQtXgpIZMxQAAJxAAE05oAAHUKAAAAAEAAAAAAAAAAAAAAABirQ6AAAAAAQAAAAhwc3A6MTQwNQAAAAEAAAAAAAAAAQAAAADpPdN37FA9KVcJfmMBuD8pPcaT5jqlrMeYEOTP36Zo2AAAAAJBVE1ZUgAAAAAAAAAAAAAAZ8rWY3iaDnWNtfpvLpNaCEbKdDjrd2gQODOuKpmj1vMAAAAAGHAagAAAAAAAAAABpIZMxQAAAEDNJwYToiBR6bzElRL4ORJdXXZYO9cE3-ishQLC_fWGrPGhWrW7_UkPJWvxWdQDJBjVOHuA1Jjc94NSe91hSwEL",
        "result_xdr": "AAAAAAAAC7j_____AAAAAQAAAAAAAAAB____-gAAAAA=",
        "result_meta_xdr": "",
        "fee_meta_xdr": "",
        "memo_type": "MemoTypeMemoText",
        "signatures": [
          "pIZMxQAAAEDNJwYToiBR6bzElRL4ORJdXXZYO9cE3-ishQLC_fWGrPGhWrW7_UkPJWvxWdQDJBjVOHuA1Jjc94NSe91hSwEL"
        ]
      }
    ]
  }
}

[Shaptic]

Just getting around to reviewing this - sorry for the delay. Are you sure the noted test actually respects the limit? I see

        "ledger": 1538224

in both records, meaning it never had to look up or ingest another. So it may be that it works w/ multiple transactions w/in the same ledger, but not for multiple.

[Shaptic]

Should we have an "invalid order" error if orderRequested != "asc" or "desc"?

[sreuland]

how do you feel about the minimal approach first, having the api do some opinionated error proofing rather than more chatter? will put in the error response, but wanted to get a read on that style aspect first?

[Shaptic]

I have no strong opinion, agreed that we should keep things minimal to keep eyes on the prize - aka working mvp

[sreuland]

Just getting around to reviewing this - sorry for the delay. Are you sure the noted test actually respects the limit? I see

        "ledger": 1538224

in both records, meaning it never had to look up or ingest another. So it may be that it works w/ multiple transactions w/in the same ledger, but not for multiple.

great catch, I think have fixed with this commit. archive.GetTransactions wasn't incrementing tx order num all the time, resulted in skipping all tx's in the first ledger encoded in cursor.

[Shaptic]

Seems like we're close! Would be really good to get some test cases as a follow-up so that we can analyze exactly what behavior is expected/reproducible/working, maybe in a follow-up PR.

[Shaptic]

Wait, I'm not entirely sure this is correct. cc @stellar/horizon-committers - does anyone know how TOIDs work? I get operation order but I don't understand transaction order. Is it essentially an index into the list of transactions in a ledger?

[Shaptic]

If yes, then this isn't right because we're still w/in the same transaction.

What I was referring to in my earlier comment, @sreuland, is that we aren't using the index in this part of the code. Once we've adjusted the cursor from the URL parameters to the next active checkpoint (done), we need to adjust the cursor until we've found enough ledgers to fulfill limit, here. On Line 99 you can see ledgerSequence++, but in reality we want ledgerSequence = index.nextActive(..., (ledgerSequence + 1) / 64).

[sreuland]

that was a mistake on my part, i incremented it in wrong place, I pushed a correction to only increment the tx order on the outside of ops loop. I'm working on a unit test here to assert the expectations.

On you point about not using the index, it appears like multiple optimal paths for the queries, when just cursor and no other query params, the results list is just a sequential listing of ops or txs from that start point, and can resolve that with just toid and the ledger backend reader, there's no value to using index there right, however, when the account query parameter is included at least as here on ops, then index.nextActive needs to be used rather than sequential toid iteration.

I think I'm working on this now for #4433, the net will be refactor the account param off /operations?account= and over to /accounts/:id/operations?cursor=&limit= and will be changing archive.GetOperations to leverage index.nextActive in the way you're recommending.

[sreuland]

@Shaptic , I added tests on archive op/tx retreival , asserts what data is on the index/archive side and what's expected on the results side, may help detect if there is a gap in current impl vs. design intent.

[2opremio]

According to SEP 35 both the transaction and operation order start at 1:

Transaction Application Order: the order that the transaction was applied within the ledger where it was validated. This takes up 20 bits of the id, from bits 32 to 51, inclusive. The application order value starts at 1. The maximum supported number of transactions per operation is 1,048,575.

Operation Index: the index of the operation within its parent transaction. This takes up 12 bits of the id, from bits 52 to 63, inclusive. The operation index value starts at 1. The maximum supported number of operations per transaction is 4095.

Thus, I think that the code is correct.

However, I would suggest bookeeping an toid.ID instead of maintaining the three counters separately.

[sreuland]

Seems like we're close! Would be really good to get some test cases as a follow-up so that we can analyze exactly what behavior is expected/reproducible/working, maybe in a follow-up PR.

I think it's worth taking a bit of time to establish some test coverage here, I'll look into a unit test with mocked backend/store.

[Shaptic]

Sorry, this might be a misunderstanding on my part, but what is the benefit of wrapping ingestion into a similar structure like this for light horizon? I can't see what it offers that the ingest package doesn't already (which is meant for public consumption as-is). Basically - why this over what was present before?

[sreuland]

the objective was to see about LightHorizon(LH) programing to its own domain model, its notion of LedgerTransaction could diverge from ingest or vice versa at any point perhaps or no? LH has defined the new Archive interface which refers to TX's so the model ends up being referenced all around in LH. This allowsingest's version of model to be used, but encapsulated to here, the rest of LH does not know ingest exists, is that worthwhile as a building block? If the abstraction just adds more complexity than it's worth, I'll yank this out np, lmk, thanks!

[Shaptic]

thought about this some more, this is a good question...

its notion of LedgerTransaction could diverge from ingest or vice versa at any point perhaps or no?

I think it's a safe bet that we'll need to do this - @paulbellamy already mentioned how we should add versioning to the meta files, which would be data on top of the xdr.LedgerCloseMeta -> ingest.LedgerTransaction ingestion layer. But the changes will be purely additive.

I'm not psyched about the naming ("adaption" and "adaptation") but that's not a blocker by any means hah; we can solve that later. (It's one of the two hard things in CS, anyway.)

[Shaptic]

thanks for fixing all of the warnings 🙏

[Shaptic]