{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":547402497,"defaultBranch":"main","name":"tree-sources","ownerLogin":"waterthetrees","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2022-10-07T16:14:34.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/70111492?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1680061625.005224","currentOid":""},"activityList":{"items":[{"before":"7e59394e76280af9c7382ad165437b0969a73cbe","after":null,"ref":"refs/heads/feature/tree-sources-staging","pushedAt":"2023-03-29T03:47:05.005Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"tzinckgraf","name":null,"path":"/tzinckgraf","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/842982?s=80&v=4"}},{"before":"314325f334c94de05d3eccbe0410d9e201440381","after":"7830a73f17b0ec79a9ccdd6ad0766b4650c99219","ref":"refs/heads/main","pushedAt":"2023-03-29T03:46:56.400Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tzinckgraf","name":null,"path":"/tzinckgraf","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/842982?s=80&v=4"},"commit":{"message":"feature/tree-sources-staging (#24)\n\n* feature/tree-sources-staging\r\n\r\nAdded a new merge step to the ETL process. The new merge step takes all\r\nthe data from the geojson file and moves it into a staging table called\r\n`tree_sources_staging`. This is a way to keep the raw data from the\r\nsources into a database table.\r\n\r\nThis PR uses ogr2ogr to move data into the database. By default, data is\r\nmoved 20k rows at a time. Postgres is all lowercase, and javascript uses\r\ncamelcase, so we cannot easily use underscores in names without some\r\nbigger changes. However, that did not seem to be an issue.\r\n\r\nfeature/tree-sources-staging\r\nRemoved unused new library\r\n\r\n* feature/tree-sources-staging\r\n- added a new PG_USE_COPY argument. this improves performance by about\r\n2-3x.\r\n- cleaned up command to have shorter strings\r\n- moved PG config to a separate variable\r\n\r\n* feature/tree-sources-staging\r\n- update the ogr2ogr command to transform the sql converting camelCase\r\nto snake_case\r\n- convert database table to treedata_staging\r\n- use spawn instead of exec\r\n- truncate the staging table as the first step\r\n\r\nAdded a new line for a little clean up\r\n\r\n* feature/tree-sources-staging\r\n- updated idName to idSourceName as per the rest of the code","shortMessageHtmlLink":"feature/tree-sources-staging (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1592424529\" data-permission-text=\"Title is private\" data-url=\"https://github.com/waterthetrees/tree-sources/issues/24\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/waterthetrees/tree-sources/pull/24/hovercard\" href=\"https://github.com/waterthetrees/tree-sources/pull/24\">#24</a>)"}},{"before":"54bc17aebd8d6964af4877e10900ed9217aff576","after":"7e59394e76280af9c7382ad165437b0969a73cbe","ref":"refs/heads/feature/tree-sources-staging","pushedAt":"2023-03-29T03:39:39.461Z","pushType":"push","commitsCount":3,"pusher":{"login":"tzinckgraf","name":null,"path":"/tzinckgraf","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/842982?s=80&v=4"},"commit":{"message":"feature/tree-sources-staging\n- updated idName to idSourceName as per the rest of the code","shortMessageHtmlLink":"feature/tree-sources-staging"}},{"before":"980b233f22c4b09f647fb57d4546b2ba79775be8","after":null,"ref":"refs/heads/zoobot/fix/change-idname-to-idSourceName","pushedAt":"2023-03-27T02:07:22.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"zoobot","name":"Rose","path":"/zoobot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22104001?s=80&v=4"}},{"before":"87b0b98bd46bd81e74f482457edbfaa017fcaee2","after":"314325f334c94de05d3eccbe0410d9e201440381","ref":"refs/heads/main","pushedAt":"2023-03-27T02:07:17.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"zoobot","name":"Rose","path":"/zoobot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22104001?s=80&v=4"},"commit":{"message":"Change-idname-to-idSourceName in all files (#30)","shortMessageHtmlLink":"Change-idname-to-idSourceName in all files (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1641118049\" data-permission-text=\"Title is private\" data-url=\"https://github.com/waterthetrees/tree-sources/issues/30\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/waterthetrees/tree-sources/pull/30/hovercard\" href=\"https://github.com/waterthetrees/tree-sources/pull/30\">#30</a>)"}},{"before":null,"after":"87b0b98bd46bd81e74f482457edbfaa017fcaee2","ref":"refs/heads/zoobot/feature/save","pushedAt":"2023-03-27T01:52:11.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"zoobot","name":"Rose","path":"/zoobot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22104001?s=80&v=4"},"commit":{"message":"feature/command-line-sources-args (#23)\n\nAdd command line arguments with yargs.\r\nThe only argument so far is a --sources argument, which is a list of\r\nsources based on the source id. This is very useful for testing instead\r\nof worrying about processing all files.\r\n\r\nThe command line arguments are added to the config in a variable called\r\n`ARGS`, then later get called when pulling the sources. The change for\r\nthe different ETL steps is seamless as a result.\r\n\r\nRemove a copy-paste import\r\n\r\nfixed up a less readable variable name","shortMessageHtmlLink":"feature/command-line-sources-args (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1592414789\" data-permission-text=\"Title is private\" data-url=\"https://github.com/waterthetrees/tree-sources/issues/23\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/waterthetrees/tree-sources/pull/23/hovercard\" href=\"https://github.com/waterthetrees/tree-sources/pull/23\">#23</a>)"}},{"before":"537e9683f48b91491ca684e3f05efb20cd523bd4","after":"54bc17aebd8d6964af4877e10900ed9217aff576","ref":"refs/heads/feature/tree-sources-staging","pushedAt":"2023-03-27T01:15:33.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"tzinckgraf","name":null,"path":"/tzinckgraf","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/842982?s=80&v=4"},"commit":{"message":"feature/tree-sources-staging\n- update the ogr2ogr command to transform the sql converting camelCase\nto snake_case\n- convert database table to treedata_staging\n- use spawn instead of exec\n- truncate the staging table as the first step\n\nAdded a new line for a little clean up","shortMessageHtmlLink":"feature/tree-sources-staging"}},{"before":null,"after":"980b233f22c4b09f647fb57d4546b2ba79775be8","ref":"refs/heads/zoobot/fix/change-idname-to-idSourceName","pushedAt":"2023-03-26T22:25:31.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"zoobot","name":"Rose","path":"/zoobot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22104001?s=80&v=4"},"commit":{"message":"Change-idname-to-idSourceName in all files","shortMessageHtmlLink":"Change-idname-to-idSourceName in all files"}},{"before":null,"after":"80349c45e8e5e9207e390677be66b4eb8461663d","ref":"refs/heads/zoobot/feature/lint","pushedAt":"2023-03-26T22:20:12.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"zoobot","name":"Rose","path":"/zoobot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22104001?s=80&v=4"},"commit":{"message":"Added .eslintrc.cjs and .prettierrc.cjs, removed .eslintrc.js","shortMessageHtmlLink":"Added .eslintrc.cjs and .prettierrc.cjs, removed .eslintrc.js"}},{"before":null,"after":"a55faec107c2f066e417a9018a132988236b98a6","ref":"refs/heads/zoobot/fix/idSourceName","pushedAt":"2023-03-26T22:15:37.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"zoobot","name":"Rose","path":"/zoobot","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22104001?s=80&v=4"},"commit":{"message":"idname changed to idSourceName","shortMessageHtmlLink":"idname changed to idSourceName"}},{"before":null,"after":"45a96638038012942cdef5413ac0d8deb857f69f","ref":"refs/heads/feature/clustering","pushedAt":"2023-03-22T02:32:18.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"tzinckgraf","name":null,"path":"/tzinckgraf","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/842982?s=80&v=4"},"commit":{"message":"feature/clustering\nAdded a clustering feature to the tile generator.\nTiles will be generated in 3 pieces. The outer zoom is one cluster /\ncity / tile, the middle zoom is regular clustering, and the inner zoom\nhas no clustering, but instead shows all points.\n\nThis is a WIP. The outer layer should actually be a geojson layer so\nthat we can cluster the clusters. The inner layer needs more zoom levels\nas it cuts off at 15.\n\nAs we are using mbtiles and geojson, we can combine the middle and inner\nmbtiles into one big mbtile, then we can combine the outer zoom at one\nlevel to create a geojson of the features at 1 / city.\n\nThis was tested with tippecanoe v2.23.0","shortMessageHtmlLink":"feature/clustering"}}],"hasNextPage":false,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAADDW_7hAA","startCursor":null,"endCursor":null}},"title":"Activity · waterthetrees/tree-sources"}