update comment in test units #3018
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This test still fails in the GH Action with the error: |
karlhigley
approved these changes
Feb 13, 2020
LaRiffle
added a commit
that referenced
this pull request
Feb 20, 2020
* Migrate CI tests to Github Actions (#2936) * Create and test pysyft package on various Pythons Includes Python 2.7, 3.5, 3.6, 3.7 * Rework build script to run tests like Travis * Update Python versions to 3.6+ * Remove Python 3.8 since Torch doesn't support it * Only run the tutorial translation tests when the tutorials change The normal tutorial notebook tests still run on every build, which should catch the cases where changes to the code break the notebooks. When the main tutorial notebooks are changed, that will trigger the action that runs the notebook translation tests, which should surface the same failures in the whichever translated versions of the tutorials are affected. * Measure process time instead of wall time in efficiency tests This counts only 'user time' and excludes 'system time', and measures relative to the CPU clock speed, which should make the tests less flaky. * Increase `max_time` for `test_inv_sym` closer to original value * Rework "test all notebooks" tests to split off translation tests * Create a separate section in the action logs for coverage reports * Exclude translation tests from coverage checks * Bump `test_inv_sym` execution time limit to 30 * Omit `.github` directory from code coverage checking * Fix translation notebooks test to account for excluded notebooks * Move coverage exclusions into setup.cfg instead of .travis.yml This way they work everywhere that coverage runs, not just on Travis. * Omit the efficiency tests from Github Action check suite Omits them from both the tests and the test coverage checks * Unwind changes to the efficiency tests (now skipped in check suite) * Stop using efficiency test time assertions outside the efficiency tests * Standardize the name of the efficiency tests module * Fix efficiency assertion imports * Fix a missing wildcard in the coverage omission flag * Apply testing workflow to PR creation, synchronization, and reopening * Clean up coverage check inclusions and exclusions * Fix triggering events section * feat: Translation to Spanish of Part 08 - Introduction to Plans. (#2962) Changes made by @socd06 This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: #2775 See also: #2782 * Bump PyTorch to 1.4 (#2930) Co-authored-by: Théo Ryffel <[email protected]> * Modify conv implementation and corresponding test to get exact match (#2964) * Modify conv implementation and corresponding test to get exact match * Revert mkldnn to the original state after execution * Exorcise a Python `eval` and transmute into the lesser demon `getattr` (#2951) * Exorcise a Python `eval` and transmute into the lesser demon `getattr` * Remove the double split for optimization * Point to correct version (#2968) * Refactor Plan internals using PlaceHolder (#2910) * Add experiment about Plans with placeholders * Add first implementation of plan using placeholders * Update notebook of test * Improve local execution of plans * Support remote execution of plans * Rm unused function * Use Operation object in plan * Remove fixed precision and sharing from Plan's multiple fetch test These seem extraneous to the point of the test anyway, so may as well go ahead and remove them rather than disabling this test. * Disable encrypted and fixed precision Plans * Remove unused `plan` attribute on `State` * Make `test_fetch_plan_multiple_times` pass * Fix typo in test name * Update serde_full unit tests (WIP) * Add example of Plan problem * Update serde unit tests * Fix output handling in plans * Fix nested Plans handling * Apply formatting to serde helpers * Adjust serde tests to match the new plan structure * Update requirements.txt to avoid a merge conflict * Undo some changes to the Torch hook * Fix Long vs Float issue in `PointerPlan` test * Remove current PromiseTensor * Remove plan.procedure * Add Trace object to clean code * Rm promise notebook * Add support for fix_precision & MPC * Clean plan code and add a tracer decorator * Clean state and rm useless functions * Clean the tracer decorator and comment code * Add comments to Plan & State new methods * Fix ordering of arguments in Plan * Fix dummy error and fix wraps decorators * Fix * Add a few comments * Merge input/output placeholders and do light cleaning * Rm useless experimental notebook * Fix serde * Increase delay in a time test Co-authored-by: Karl Higley <[email protected]> Co-authored-by: Vova Manannikov <[email protected]> * add Part 13a Hindi (#2958) Co-authored-by: Andrew Trask <[email protected]> * Bump version to `0.2.3.a1` (#2976) * Refactored code of SplitNN model and added evaluation on datasets (#2983) * Add Protobuf serialization for Placeholders, Plans, and States (#2972) * Add Protobuf serialization for Placeholders, Plans, and States * Bump `syft-proto` requirement to `0.1.1.a1.post5` Co-authored-by: Andrew Trask <[email protected]> * Make it possible to JIT trace a PySyft Plan to convert to Torchscript (#2988) * updated requirements.txt (#2970) * updated requirements.txt * Update requirements.txt Co-authored-by: George-Cristian Muraru <[email protected]> * Translation to Spanish of Tutorial Part 12 bis (#2979) feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: #2775 See also: #2782 Co-authored-by: Karl Higley <[email protected]> * Remove codefix for old pytorch version (#2999) * feat: Translation to Spanish of part 08 bis (#3011) This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: Track better the changes in every document. Avoid double work. Resolves: #2775 See also: #2782 * Added Bengali Translated Notebooks (#2992) Translated Notebooks: - Part 13a - Secure Classification with Syft Keras and TFE - Public Training.ipynb - Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb - Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Add Udacity Link to Readme (#3016) * Add sphinx documentation with automatic API building (#3017) * Add autoapi.extension to sphinx configuration * Add autoapi as dev dep. Add doc build to travis * Add doc testenv to github actions and travis * Fix formatting via black * Add readthedocs yml configuration * Configure sphinx to ignore "_static" * Update theme to sphinx-rtd-theme * Update index.rst Co-authored-by: Théo Ryffel <[email protected]> * Adding some notebooks translated to Portuguese (#3015) * Adding Part 12 tutorial in portuguese * Adding Parte13a tutorial in portuguese Co-authored-by: Andrew Trask <[email protected]> * trigger github action notebook checks (#3012) * use doubel qoutes instead * edit notebook just to trigger testing * add pull_request trigger condition * Revert "edit notebook just to trigger testing" This reverts commit 7e46d5b. * add path to pull_request * edit notebook again just to see if checks trigger * rename notebook .yml file * edit name string * change pull_request to master branch condition * master branch trigger condition * Revert "edit notebook again just to see if checks trigger" This reverts commit 60fee72. * small edit just to trigger notebook tests, AGAIN! * Revert "small edit just to trigger notebook tests, AGAIN!" This reverts commit 16a83d2. * Revert changes to `run-tests.yaml` This one is already working the way we want. * Make action workflows match Co-authored-by: Karl Higley <[email protected]> * Translation into Ukrainian language (#2854) * Add files via upload * Update Part 01 - Основні засоби приватного глибинного навчання.ipynb Co-authored-by: Karl Higley <[email protected]> Co-authored-by: George-Cristian Muraru <[email protected]> Co-authored-by: Théo Ryffel <[email protected]> * Remove travis (#3013) * delete travis.yml * remove travis from contributing.md * remoev travis from readme * add note for future contributers in test_notebooks * add github actions badge * edit typo * add notebook GH test badge Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Hindi translations for Part 5, 6, 13b and 13c (#2909) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * (minor fix) fix broked badge for run notebook tests (#3028) * fix: failing tutorial translations should be ignored (#3034) * Add Bengali Translated Notebooks #Part 12 bis (#3038) Translated Notebooks: Part 12 bis - Encrypted Training on MNIST.ipynb * Add Bengali Translated Notebooks #Part 12 (#3039) Translated Notebooks: Part 12 - Train an Encrypted Neural Network on Encrypted Data.ipynb * fix: failing tutorial translations should be ignored (#3047) Exclude only the names, instead of full paths. This means it will effectively exclude translated titles too. * Translation to Portuguese of the tutorials Part 09, 10, and 11 (#2980) * add translation to Portuguese of the tutorial Part 02 * (enhance) translate example toy expression literally * (minor fixes) change 'portugues' to 'português' and 'nível superior' to 'parte superior' * add translation to Portuguese of the tutorial Part 03 * (fix) remove old folder name * add translation to Portuguese of the tutorial Part 04 * (update) rename 'portugues' to 'português' and remove old folder * add translation to Portuguese of the tutorial Part 05 * add translation to Portuguese of the tutorial Part 06 * (minor fixes) typo and verb correction * add translation to Portuguese of the tutorial Part 09 * add translation to Portuguese of the tutorial Part 10 * add translation to Portuguese of the tutorial Part 11 * (fix) a cell has been removed by mistake 😑 * (minor fixes) 🤞 🤞 🤞 * (minor fix) full stop added * (fixes) remove repeated paragraphs left in tutorials part 06 and 11 Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: George-Cristian Muraru <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Portuguese translations Part7-8-8bis (#2977) * pt translation part07 * middle part 8 translation * fixing portuges * finishing translation part8 and part8 bis * fixing errors * finishing tutorial part 8bis * changing metadata kernel to python3 * update parte07-08 - fixing some errors * Update Parte 07 - Aprendizado Federado com Conjunto de Dados Federado.ipynb * Update Parte 08 - Introdução a Planos.ipynb Co-authored-by: Karl Higley <[email protected]> * Translated tutorial part 01 to Portuguese (#3035) Co-authored-by: Karl Higley <[email protected]> * Translate PySyft Tutorials to Portuguese (#3020) * Translate notebook Part 13b * translate notebook part 12 bis to portuguese * Update Parte 12 bis - Treinamento criptografado no MNIST.ipynb Co-authored-by: Karl Higley <[email protected]> * Portuguese translation 13c (#3023) * translate 13c * translate 13c update * translate 13c update 2 * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb Co-authored-by: Karl Higley <[email protected]> * No need for requires_grad for input data (#2998) * No need for requires_grad for input data * Increase error threshold Co-authored-by: Andrew Trask <[email protected]> * Put the test badges first in README (#3046) * Put the test badges first * Update the names of test workflows to clean up README badges Co-authored-by: Théo Ryffel <[email protected]> Co-authored-by: Andrew Trask <[email protected]> * Add Bengali Translated Notebooks #Part 11 (#3041) Translated Notebook: Part 11 - Secure Deep Learning Classification.ipynb Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Add instruction for PySyft-Proto dependency and numbered the steps (#2818) * Add instruction for PySyft-Proto dependency and numbered the steps * Remove Step 5, it is handled in requirement_dev.txt * Update requirements files to point to syft-proto in PyPI * Remove duplicate dependency (syft-proto) outside requirements.txt * replace proto with syft-proto * Cleanup PATE code (#3033) * Format PATE docstrings Uniform indentation First pass of adding type hints Remove commented out TF flags code * Remove commented out print statements in PATE code * Change blank lines in PATE code No blank lines after function doc strings No blank lines immediately after control statements Break up larger chunks of code * Specify l and noise_eps when printing if cant compute sensitivity Before this commit, incorrect noise/moments spammed console with identical statements * Change torch/numpy sensitivity function type hints * Add notice required under Apache 2.0 license that changes have been made to code * Remove ellipses from List in type hints Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * update comment in test units (#3018) * remove test unit * Add a missing comma that `black` complains about * add ignore test back and update comment. Co-authored-by: Karl Higley <[email protected]> * Fix potential medium/high severity security holes (#3037) * remove security warning messages * Fix missing ast import * Flag the `eval` in `Plan` so that `bandit` security scan ignores it We do want to fix this and find a better way to do it, but don't want it to block adding security scans to, for example, catch future `eval`s added to the code base. * Black formatter wants two spaces before a comment Co-authored-by: Karl Higley <[email protected]> Co-authored-by: Andrew Trask <[email protected]> * CI security testing (#3036) * add bandit as dev dependancy * add job for bandit * only report medium and high severity * Match format of security task name with the others Co-authored-by: Karl Higley <[email protected]> * Only test translation notebooks if they have been changed (#3060) * Only test translated tutorials that have been changed * Add test_notebooks_basic_translations_diff for GHActions * Change GitHub Actions test for translations * Include git-diff file * Tutorial Notebook 10 note on numpy hook (#3022) * added note about notebook not working * phrasing Co-authored-by: Karl Higley <[email protected]> * Revert "Only test translation notebooks if they have been changed (#3060)" (#3062) This reverts commit 91c8114. * Split tutorial translation tests into separate Github Workflows (#3059) * Split tutorial translation tests into separate Github Workflows * Update the main tests workflow to exclude translation tests * Lower-case the directory name for Ukrainian translations * Add update-docker-image.yml to GitHub Actions workflow (#3030) * Remove pointer tags/description decode instructions (#3067) * Update GH checkout action to v2 (#3068) * Fix trigger paths in translation test workflows (#3065) * Fix trigger paths in translation test workflows * Update GH checkout action to v2 * Correct build context for dockerhub update via GitHub Actions (#3066) * fix build context for dockerhub * Update GH checkout action to v2 Co-authored-by: Karl Higley <[email protected]> * Update msgpack version requirement (#3073) * Reduce websocket timeout to one minute (#3072) This is making CI tests hang ~forever. I think 278 hours is a bit excessive, don't you? :) * Add wildcards to the tutorial translation test actions (#3071) * Translation to Spanish of Parts 09, 13 (a, b and c) (#3032) * feat: Translation to Spanish of Parts 09, 13 (a, b and c). Credits to: @thefirebanks This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: #2775 See also: #2782 * Fix typos in Spanish tutorials parts 9 and 13b Co-authored-by: Karl Higley <[email protected]> * Translations to Spanish of Tutorials Part 10 and 11 (#3031) * feat: Translation to Spanish of Part 10 - Federated Learning with Secure Aggregation. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: #2775 See also: #2782 * feat: Translation to Spanish of Part 11 - Secure Deep Learning Classification This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: #2775 See also: #2782 * Fix typos in Spanish translations parts 10 and 11 Co-authored-by: Karl Higley <[email protected]> * Notebook 7 hindi translation (#3055) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 hindi translation * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 - hindi translation Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Bengali 6 (#3052) * Bengali 6 Added * Fix markdown link syntax for OpenMined blog post Co-authored-by: Karl Higley <[email protected]> * Bengali 7 (#3053) * Bengali 7 Added * Fix link to PySyft project issues Co-authored-by: Karl Higley <[email protected]> * Split the (English) tutorial tests into a separate Github Action (#3075) * Split the (English) tutorial tests into a separate Github Action The tutorial tests take about 15 minutes to run, which is approximately equal to the running time of all the rest of the tests. Running them in parallel should speed up the PR checks. * Exclude the isolated file system test fixture from coverage checks It's only used by the notebook tests, which are run separately as of this pull request. * Run notebook tests on all PRs except translation tests Before this change, they would only run on PRs that changed notebooks. * Context of computation for crypten (#2963) * New message type for crypten party initialization * Handle CryptenInit message by running local party * Define a new context of computation for crypten The new context of computation can run parties that are distributed across syft workers. The communication of crypten parties remains the same, however, syft workers handle initialization and the serialization of return values. * updated docs * testing crypten context * fix: was setting the bad env variable DISTRIBUTED_BACKEND should have been set and not BACKEND * test serde of CryptenInit message * add crypten as core deps, this should change to extra * delete useless comment * fix: list CryptenInit in OBJ_SIMPLIFIER_AND_DETAILERS * fix msgpack tests * run black * don't cover empty function * Use tag for crypten data (#3003) Co-authored-by: Karl Higley <[email protected]> Co-authored-by: arturomf94 <[email protected]> Co-authored-by: Théo Ryffel <[email protected]> Co-authored-by: Arshjot Singh Khehra <[email protected]> Co-authored-by: Vova Manannikov <[email protected]> Co-authored-by: Yugandhar Tripathi <[email protected]> Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: midokura-silvia <[email protected]> Co-authored-by: Carlos Salgado <[email protected]> Co-authored-by: Ricardo Pretelt <[email protected]> Co-authored-by: Sourav Das <[email protected]> Co-authored-by: Aiden V Johnson <[email protected]> Co-authored-by: José Benardi de Souza Nunes <[email protected]> Co-authored-by: Marcus Vinicius <[email protected]> Co-authored-by: systemshift <[email protected]> Co-authored-by: Bogdan Ivanyuk <[email protected]> Co-authored-by: Urvashi Bhattacharyya <[email protected]> Co-authored-by: Jeferson Ferreira <[email protected]> Co-authored-by: João Lucas <[email protected]> Co-authored-by: Márcio Porto <[email protected]> Co-authored-by: Héricles Emanuel <[email protected]> Co-authored-by: Izabella Antonino <[email protected]> Co-authored-by: Bolarinwa Saheed Olayemi <[email protected]> Co-authored-by: Tom <[email protected]> Co-authored-by: froessler <[email protected]> Co-authored-by: Lina Tran <[email protected]> Co-authored-by: Ionésio Junior <[email protected]> Co-authored-by: Henning Dodenhof <[email protected]> Co-authored-by: Sayantan Das <[email protected]> Co-authored-by: Ayoub Benaissa <[email protected]>
gmuraru
added a commit
to gmuraru/PySyft
that referenced
this pull request
Mar 3, 2020
* Migrate CI tests to Github Actions (OpenMined#2936) * Create and test pysyft package on various Pythons Includes Python 2.7, 3.5, 3.6, 3.7 * Rework build script to run tests like Travis * Update Python versions to 3.6+ * Remove Python 3.8 since Torch doesn't support it * Only run the tutorial translation tests when the tutorials change The normal tutorial notebook tests still run on every build, which should catch the cases where changes to the code break the notebooks. When the main tutorial notebooks are changed, that will trigger the action that runs the notebook translation tests, which should surface the same failures in the whichever translated versions of the tutorials are affected. * Measure process time instead of wall time in efficiency tests This counts only 'user time' and excludes 'system time', and measures relative to the CPU clock speed, which should make the tests less flaky. * Increase `max_time` for `test_inv_sym` closer to original value * Rework "test all notebooks" tests to split off translation tests * Create a separate section in the action logs for coverage reports * Exclude translation tests from coverage checks * Bump `test_inv_sym` execution time limit to 30 * Omit `.github` directory from code coverage checking * Fix translation notebooks test to account for excluded notebooks * Move coverage exclusions into setup.cfg instead of .travis.yml This way they work everywhere that coverage runs, not just on Travis. * Omit the efficiency tests from Github Action check suite Omits them from both the tests and the test coverage checks * Unwind changes to the efficiency tests (now skipped in check suite) * Stop using efficiency test time assertions outside the efficiency tests * Standardize the name of the efficiency tests module * Fix efficiency assertion imports * Fix a missing wildcard in the coverage omission flag * Apply testing workflow to PR creation, synchronization, and reopening * Clean up coverage check inclusions and exclusions * Fix triggering events section * feat: Translation to Spanish of Part 08 - Introduction to Plans. (OpenMined#2962) Changes made by @socd06 This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Bump PyTorch to 1.4 (OpenMined#2930) Co-authored-by: Théo Ryffel <[email protected]> * Modify conv implementation and corresponding test to get exact match (OpenMined#2964) * Modify conv implementation and corresponding test to get exact match * Revert mkldnn to the original state after execution * Exorcise a Python `eval` and transmute into the lesser demon `getattr` (OpenMined#2951) * Exorcise a Python `eval` and transmute into the lesser demon `getattr` * Remove the double split for optimization * Point to correct version (OpenMined#2968) * Refactor Plan internals using PlaceHolder (OpenMined#2910) * Add experiment about Plans with placeholders * Add first implementation of plan using placeholders * Update notebook of test * Improve local execution of plans * Support remote execution of plans * Rm unused function * Use Operation object in plan * Remove fixed precision and sharing from Plan's multiple fetch test These seem extraneous to the point of the test anyway, so may as well go ahead and remove them rather than disabling this test. * Disable encrypted and fixed precision Plans * Remove unused `plan` attribute on `State` * Make `test_fetch_plan_multiple_times` pass * Fix typo in test name * Update serde_full unit tests (WIP) * Add example of Plan problem * Update serde unit tests * Fix output handling in plans * Fix nested Plans handling * Apply formatting to serde helpers * Adjust serde tests to match the new plan structure * Update requirements.txt to avoid a merge conflict * Undo some changes to the Torch hook * Fix Long vs Float issue in `PointerPlan` test * Remove current PromiseTensor * Remove plan.procedure * Add Trace object to clean code * Rm promise notebook * Add support for fix_precision & MPC * Clean plan code and add a tracer decorator * Clean state and rm useless functions * Clean the tracer decorator and comment code * Add comments to Plan & State new methods * Fix ordering of arguments in Plan * Fix dummy error and fix wraps decorators * Fix * Add a few comments * Merge input/output placeholders and do light cleaning * Rm useless experimental notebook * Fix serde * Increase delay in a time test Co-authored-by: Karl Higley <[email protected]> Co-authored-by: Vova Manannikov <[email protected]> * add Part 13a Hindi (OpenMined#2958) Co-authored-by: Andrew Trask <[email protected]> * Bump version to `0.2.3.a1` (OpenMined#2976) * Refactored code of SplitNN model and added evaluation on datasets (OpenMined#2983) * Add Protobuf serialization for Placeholders, Plans, and States (OpenMined#2972) * Add Protobuf serialization for Placeholders, Plans, and States * Bump `syft-proto` requirement to `0.1.1.a1.post5` Co-authored-by: Andrew Trask <[email protected]> * Make it possible to JIT trace a PySyft Plan to convert to Torchscript (OpenMined#2988) * updated requirements.txt (OpenMined#2970) * updated requirements.txt * Update requirements.txt Co-authored-by: George-Cristian Muraru <[email protected]> * Translation to Spanish of Tutorial Part 12 bis (OpenMined#2979) feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 Co-authored-by: Karl Higley <[email protected]> * Remove codefix for old pytorch version (OpenMined#2999) * feat: Translation to Spanish of part 08 bis (OpenMined#3011) This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: Track better the changes in every document. Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Added Bengali Translated Notebooks (OpenMined#2992) Translated Notebooks: - Part 13a - Secure Classification with Syft Keras and TFE - Public Training.ipynb - Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb - Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Add Udacity Link to Readme (OpenMined#3016) * Add sphinx documentation with automatic API building (OpenMined#3017) * Add autoapi.extension to sphinx configuration * Add autoapi as dev dep. Add doc build to travis * Add doc testenv to github actions and travis * Fix formatting via black * Add readthedocs yml configuration * Configure sphinx to ignore "_static" * Update theme to sphinx-rtd-theme * Update index.rst Co-authored-by: Théo Ryffel <[email protected]> * Adding some notebooks translated to Portuguese (OpenMined#3015) * Adding Part 12 tutorial in portuguese * Adding Parte13a tutorial in portuguese Co-authored-by: Andrew Trask <[email protected]> * trigger github action notebook checks (OpenMined#3012) * use doubel qoutes instead * edit notebook just to trigger testing * add pull_request trigger condition * Revert "edit notebook just to trigger testing" This reverts commit 7e46d5b. * add path to pull_request * edit notebook again just to see if checks trigger * rename notebook .yml file * edit name string * change pull_request to master branch condition * master branch trigger condition * Revert "edit notebook again just to see if checks trigger" This reverts commit 60fee72. * small edit just to trigger notebook tests, AGAIN! * Revert "small edit just to trigger notebook tests, AGAIN!" This reverts commit 16a83d2. * Revert changes to `run-tests.yaml` This one is already working the way we want. * Make action workflows match Co-authored-by: Karl Higley <[email protected]> * Translation into Ukrainian language (OpenMined#2854) * Add files via upload * Update Part 01 - Основні засоби приватного глибинного навчання.ipynb Co-authored-by: Karl Higley <[email protected]> Co-authored-by: George-Cristian Muraru <[email protected]> Co-authored-by: Théo Ryffel <[email protected]> * Remove travis (OpenMined#3013) * delete travis.yml * remove travis from contributing.md * remoev travis from readme * add note for future contributers in test_notebooks * add github actions badge * edit typo * add notebook GH test badge Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Hindi translations for Part 5, 6, 13b and 13c (OpenMined#2909) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * (minor fix) fix broked badge for run notebook tests (OpenMined#3028) * fix: failing tutorial translations should be ignored (OpenMined#3034) * Add Bengali Translated Notebooks #Part 12 bis (OpenMined#3038) Translated Notebooks: Part 12 bis - Encrypted Training on MNIST.ipynb * Add Bengali Translated Notebooks #Part 12 (OpenMined#3039) Translated Notebooks: Part 12 - Train an Encrypted Neural Network on Encrypted Data.ipynb * fix: failing tutorial translations should be ignored (OpenMined#3047) Exclude only the names, instead of full paths. This means it will effectively exclude translated titles too. * Translation to Portuguese of the tutorials Part 09, 10, and 11 (OpenMined#2980) * add translation to Portuguese of the tutorial Part 02 * (enhance) translate example toy expression literally * (minor fixes) change 'portugues' to 'português' and 'nível superior' to 'parte superior' * add translation to Portuguese of the tutorial Part 03 * (fix) remove old folder name * add translation to Portuguese of the tutorial Part 04 * (update) rename 'portugues' to 'português' and remove old folder * add translation to Portuguese of the tutorial Part 05 * add translation to Portuguese of the tutorial Part 06 * (minor fixes) typo and verb correction * add translation to Portuguese of the tutorial Part 09 * add translation to Portuguese of the tutorial Part 10 * add translation to Portuguese of the tutorial Part 11 * (fix) a cell has been removed by mistake 😑 * (minor fixes) 🤞 🤞 🤞 * (minor fix) full stop added * (fixes) remove repeated paragraphs left in tutorials part 06 and 11 Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: George-Cristian Muraru <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Portuguese translations Part7-8-8bis (OpenMined#2977) * pt translation part07 * middle part 8 translation * fixing portuges * finishing translation part8 and part8 bis * fixing errors * finishing tutorial part 8bis * changing metadata kernel to python3 * update parte07-08 - fixing some errors * Update Parte 07 - Aprendizado Federado com Conjunto de Dados Federado.ipynb * Update Parte 08 - Introdução a Planos.ipynb Co-authored-by: Karl Higley <[email protected]> * Translated tutorial part 01 to Portuguese (OpenMined#3035) Co-authored-by: Karl Higley <[email protected]> * Translate PySyft Tutorials to Portuguese (OpenMined#3020) * Translate notebook Part 13b * translate notebook part 12 bis to portuguese * Update Parte 12 bis - Treinamento criptografado no MNIST.ipynb Co-authored-by: Karl Higley <[email protected]> * Portuguese translation 13c (OpenMined#3023) * translate 13c * translate 13c update * translate 13c update 2 * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb Co-authored-by: Karl Higley <[email protected]> * No need for requires_grad for input data (OpenMined#2998) * No need for requires_grad for input data * Increase error threshold Co-authored-by: Andrew Trask <[email protected]> * Put the test badges first in README (OpenMined#3046) * Put the test badges first * Update the names of test workflows to clean up README badges Co-authored-by: Théo Ryffel <[email protected]> Co-authored-by: Andrew Trask <[email protected]> * Add Bengali Translated Notebooks #Part 11 (OpenMined#3041) Translated Notebook: Part 11 - Secure Deep Learning Classification.ipynb Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Add instruction for PySyft-Proto dependency and numbered the steps (OpenMined#2818) * Add instruction for PySyft-Proto dependency and numbered the steps * Remove Step 5, it is handled in requirement_dev.txt * Update requirements files to point to syft-proto in PyPI * Remove duplicate dependency (syft-proto) outside requirements.txt * replace proto with syft-proto * Cleanup PATE code (OpenMined#3033) * Format PATE docstrings Uniform indentation First pass of adding type hints Remove commented out TF flags code * Remove commented out print statements in PATE code * Change blank lines in PATE code No blank lines after function doc strings No blank lines immediately after control statements Break up larger chunks of code * Specify l and noise_eps when printing if cant compute sensitivity Before this commit, incorrect noise/moments spammed console with identical statements * Change torch/numpy sensitivity function type hints * Add notice required under Apache 2.0 license that changes have been made to code * Remove ellipses from List in type hints Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * update comment in test units (OpenMined#3018) * remove test unit * Add a missing comma that `black` complains about * add ignore test back and update comment. Co-authored-by: Karl Higley <[email protected]> * Fix potential medium/high severity security holes (OpenMined#3037) * remove security warning messages * Fix missing ast import * Flag the `eval` in `Plan` so that `bandit` security scan ignores it We do want to fix this and find a better way to do it, but don't want it to block adding security scans to, for example, catch future `eval`s added to the code base. * Black formatter wants two spaces before a comment Co-authored-by: Karl Higley <[email protected]> Co-authored-by: Andrew Trask <[email protected]> * CI security testing (OpenMined#3036) * add bandit as dev dependancy * add job for bandit * only report medium and high severity * Match format of security task name with the others Co-authored-by: Karl Higley <[email protected]> * Only test translation notebooks if they have been changed (OpenMined#3060) * Only test translated tutorials that have been changed * Add test_notebooks_basic_translations_diff for GHActions * Change GitHub Actions test for translations * Include git-diff file * Tutorial Notebook 10 note on numpy hook (OpenMined#3022) * added note about notebook not working * phrasing Co-authored-by: Karl Higley <[email protected]> * Revert "Only test translation notebooks if they have been changed (OpenMined#3060)" (OpenMined#3062) This reverts commit 91c8114. * Split tutorial translation tests into separate Github Workflows (OpenMined#3059) * Split tutorial translation tests into separate Github Workflows * Update the main tests workflow to exclude translation tests * Lower-case the directory name for Ukrainian translations * Add update-docker-image.yml to GitHub Actions workflow (OpenMined#3030) * Remove pointer tags/description decode instructions (OpenMined#3067) * Update GH checkout action to v2 (OpenMined#3068) * Fix trigger paths in translation test workflows (OpenMined#3065) * Fix trigger paths in translation test workflows * Update GH checkout action to v2 * Correct build context for dockerhub update via GitHub Actions (OpenMined#3066) * fix build context for dockerhub * Update GH checkout action to v2 Co-authored-by: Karl Higley <[email protected]> * Update msgpack version requirement (OpenMined#3073) * Reduce websocket timeout to one minute (OpenMined#3072) This is making CI tests hang ~forever. I think 278 hours is a bit excessive, don't you? :) * Add wildcards to the tutorial translation test actions (OpenMined#3071) * Translation to Spanish of Parts 09, 13 (a, b and c) (OpenMined#3032) * feat: Translation to Spanish of Parts 09, 13 (a, b and c). Credits to: @thefirebanks This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Fix typos in Spanish tutorials parts 9 and 13b Co-authored-by: Karl Higley <[email protected]> * Translations to Spanish of Tutorials Part 10 and 11 (OpenMined#3031) * feat: Translation to Spanish of Part 10 - Federated Learning with Secure Aggregation. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * feat: Translation to Spanish of Part 11 - Secure Deep Learning Classification This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Fix typos in Spanish translations parts 10 and 11 Co-authored-by: Karl Higley <[email protected]> * Notebook 7 hindi translation (OpenMined#3055) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 hindi translation * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 - hindi translation Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Bengali 6 (OpenMined#3052) * Bengali 6 Added * Fix markdown link syntax for OpenMined blog post Co-authored-by: Karl Higley <[email protected]> * Bengali 7 (OpenMined#3053) * Bengali 7 Added * Fix link to PySyft project issues Co-authored-by: Karl Higley <[email protected]> * Split the (English) tutorial tests into a separate Github Action (OpenMined#3075) * Split the (English) tutorial tests into a separate Github Action The tutorial tests take about 15 minutes to run, which is approximately equal to the running time of all the rest of the tests. Running them in parallel should speed up the PR checks. * Exclude the isolated file system test fixture from coverage checks It's only used by the notebook tests, which are run separately as of this pull request. * Run notebook tests on all PRs except translation tests Before this change, they would only run on PRs that changed notebooks. * Context of computation for crypten (OpenMined#2963) * New message type for crypten party initialization * Handle CryptenInit message by running local party * Define a new context of computation for crypten The new context of computation can run parties that are distributed across syft workers. The communication of crypten parties remains the same, however, syft workers handle initialization and the serialization of return values. * updated docs * testing crypten context * fix: was setting the bad env variable DISTRIBUTED_BACKEND should have been set and not BACKEND * test serde of CryptenInit message * add crypten as core deps, this should change to extra * delete useless comment * fix: list CryptenInit in OBJ_SIMPLIFIER_AND_DETAILERS * fix msgpack tests * run black * don't cover empty function * Use tag for crypten data (OpenMined#3003) Co-authored-by: Karl Higley <[email protected]> Co-authored-by: arturomf94 <[email protected]> Co-authored-by: Théo Ryffel <[email protected]> Co-authored-by: Arshjot Singh Khehra <[email protected]> Co-authored-by: Vova Manannikov <[email protected]> Co-authored-by: Yugandhar Tripathi <[email protected]> Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: midokura-silvia <[email protected]> Co-authored-by: Carlos Salgado <[email protected]> Co-authored-by: Ricardo Pretelt <[email protected]> Co-authored-by: Sourav Das <[email protected]> Co-authored-by: Aiden V Johnson <[email protected]> Co-authored-by: José Benardi de Souza Nunes <[email protected]> Co-authored-by: Marcus Vinicius <[email protected]> Co-authored-by: systemshift <[email protected]> Co-authored-by: Bogdan Ivanyuk <[email protected]> Co-authored-by: Urvashi Bhattacharyya <[email protected]> Co-authored-by: Jeferson Ferreira <[email protected]> Co-authored-by: João Lucas <[email protected]> Co-authored-by: Márcio Porto <[email protected]> Co-authored-by: Héricles Emanuel <[email protected]> Co-authored-by: Izabella Antonino <[email protected]> Co-authored-by: Bolarinwa Saheed Olayemi <[email protected]> Co-authored-by: Tom <[email protected]> Co-authored-by: froessler <[email protected]> Co-authored-by: Lina Tran <[email protected]> Co-authored-by: Ionésio Junior <[email protected]> Co-authored-by: Henning Dodenhof <[email protected]> Co-authored-by: Sayantan Das <[email protected]> Co-authored-by: Ayoub Benaissa <[email protected]>
gmuraru
added a commit
to gmuraru/PySyft
that referenced
this pull request
Mar 16, 2020
* Migrate CI tests to Github Actions (OpenMined#2936) * Create and test pysyft package on various Pythons Includes Python 2.7, 3.5, 3.6, 3.7 * Rework build script to run tests like Travis * Update Python versions to 3.6+ * Remove Python 3.8 since Torch doesn't support it * Only run the tutorial translation tests when the tutorials change The normal tutorial notebook tests still run on every build, which should catch the cases where changes to the code break the notebooks. When the main tutorial notebooks are changed, that will trigger the action that runs the notebook translation tests, which should surface the same failures in the whichever translated versions of the tutorials are affected. * Measure process time instead of wall time in efficiency tests This counts only 'user time' and excludes 'system time', and measures relative to the CPU clock speed, which should make the tests less flaky. * Increase `max_time` for `test_inv_sym` closer to original value * Rework "test all notebooks" tests to split off translation tests * Create a separate section in the action logs for coverage reports * Exclude translation tests from coverage checks * Bump `test_inv_sym` execution time limit to 30 * Omit `.github` directory from code coverage checking * Fix translation notebooks test to account for excluded notebooks * Move coverage exclusions into setup.cfg instead of .travis.yml This way they work everywhere that coverage runs, not just on Travis. * Omit the efficiency tests from Github Action check suite Omits them from both the tests and the test coverage checks * Unwind changes to the efficiency tests (now skipped in check suite) * Stop using efficiency test time assertions outside the efficiency tests * Standardize the name of the efficiency tests module * Fix efficiency assertion imports * Fix a missing wildcard in the coverage omission flag * Apply testing workflow to PR creation, synchronization, and reopening * Clean up coverage check inclusions and exclusions * Fix triggering events section * feat: Translation to Spanish of Part 08 - Introduction to Plans. (OpenMined#2962) Changes made by @socd06 This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Bump PyTorch to 1.4 (OpenMined#2930) Co-authored-by: Théo Ryffel <[email protected]> * Modify conv implementation and corresponding test to get exact match (OpenMined#2964) * Modify conv implementation and corresponding test to get exact match * Revert mkldnn to the original state after execution * Exorcise a Python `eval` and transmute into the lesser demon `getattr` (OpenMined#2951) * Exorcise a Python `eval` and transmute into the lesser demon `getattr` * Remove the double split for optimization * Point to correct version (OpenMined#2968) * Refactor Plan internals using PlaceHolder (OpenMined#2910) * Add experiment about Plans with placeholders * Add first implementation of plan using placeholders * Update notebook of test * Improve local execution of plans * Support remote execution of plans * Rm unused function * Use Operation object in plan * Remove fixed precision and sharing from Plan's multiple fetch test These seem extraneous to the point of the test anyway, so may as well go ahead and remove them rather than disabling this test. * Disable encrypted and fixed precision Plans * Remove unused `plan` attribute on `State` * Make `test_fetch_plan_multiple_times` pass * Fix typo in test name * Update serde_full unit tests (WIP) * Add example of Plan problem * Update serde unit tests * Fix output handling in plans * Fix nested Plans handling * Apply formatting to serde helpers * Adjust serde tests to match the new plan structure * Update requirements.txt to avoid a merge conflict * Undo some changes to the Torch hook * Fix Long vs Float issue in `PointerPlan` test * Remove current PromiseTensor * Remove plan.procedure * Add Trace object to clean code * Rm promise notebook * Add support for fix_precision & MPC * Clean plan code and add a tracer decorator * Clean state and rm useless functions * Clean the tracer decorator and comment code * Add comments to Plan & State new methods * Fix ordering of arguments in Plan * Fix dummy error and fix wraps decorators * Fix * Add a few comments * Merge input/output placeholders and do light cleaning * Rm useless experimental notebook * Fix serde * Increase delay in a time test Co-authored-by: Karl Higley <[email protected]> Co-authored-by: Vova Manannikov <[email protected]> * add Part 13a Hindi (OpenMined#2958) Co-authored-by: Andrew Trask <[email protected]> * Bump version to `0.2.3.a1` (OpenMined#2976) * Refactored code of SplitNN model and added evaluation on datasets (OpenMined#2983) * Add Protobuf serialization for Placeholders, Plans, and States (OpenMined#2972) * Add Protobuf serialization for Placeholders, Plans, and States * Bump `syft-proto` requirement to `0.1.1.a1.post5` Co-authored-by: Andrew Trask <[email protected]> * Make it possible to JIT trace a PySyft Plan to convert to Torchscript (OpenMined#2988) * updated requirements.txt (OpenMined#2970) * updated requirements.txt * Update requirements.txt Co-authored-by: George-Cristian Muraru <[email protected]> * Translation to Spanish of Tutorial Part 12 bis (OpenMined#2979) feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 Co-authored-by: Karl Higley <[email protected]> * Remove codefix for old pytorch version (OpenMined#2999) * feat: Translation to Spanish of part 08 bis (OpenMined#3011) This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: Track better the changes in every document. Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Added Bengali Translated Notebooks (OpenMined#2992) Translated Notebooks: - Part 13a - Secure Classification with Syft Keras and TFE - Public Training.ipynb - Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb - Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Add Udacity Link to Readme (OpenMined#3016) * Add sphinx documentation with automatic API building (OpenMined#3017) * Add autoapi.extension to sphinx configuration * Add autoapi as dev dep. Add doc build to travis * Add doc testenv to github actions and travis * Fix formatting via black * Add readthedocs yml configuration * Configure sphinx to ignore "_static" * Update theme to sphinx-rtd-theme * Update index.rst Co-authored-by: Théo Ryffel <[email protected]> * Adding some notebooks translated to Portuguese (OpenMined#3015) * Adding Part 12 tutorial in portuguese * Adding Parte13a tutorial in portuguese Co-authored-by: Andrew Trask <[email protected]> * trigger github action notebook checks (OpenMined#3012) * use doubel qoutes instead * edit notebook just to trigger testing * add pull_request trigger condition * Revert "edit notebook just to trigger testing" This reverts commit 7e46d5b. * add path to pull_request * edit notebook again just to see if checks trigger * rename notebook .yml file * edit name string * change pull_request to master branch condition * master branch trigger condition * Revert "edit notebook again just to see if checks trigger" This reverts commit 60fee72. * small edit just to trigger notebook tests, AGAIN! * Revert "small edit just to trigger notebook tests, AGAIN!" This reverts commit 16a83d2. * Revert changes to `run-tests.yaml` This one is already working the way we want. * Make action workflows match Co-authored-by: Karl Higley <[email protected]> * Translation into Ukrainian language (OpenMined#2854) * Add files via upload * Update Part 01 - Основні засоби приватного глибинного навчання.ipynb Co-authored-by: Karl Higley <[email protected]> Co-authored-by: George-Cristian Muraru <[email protected]> Co-authored-by: Théo Ryffel <[email protected]> * Remove travis (OpenMined#3013) * delete travis.yml * remove travis from contributing.md * remoev travis from readme * add note for future contributers in test_notebooks * add github actions badge * edit typo * add notebook GH test badge Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Hindi translations for Part 5, 6, 13b and 13c (OpenMined#2909) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * (minor fix) fix broked badge for run notebook tests (OpenMined#3028) * fix: failing tutorial translations should be ignored (OpenMined#3034) * Add Bengali Translated Notebooks #Part 12 bis (OpenMined#3038) Translated Notebooks: Part 12 bis - Encrypted Training on MNIST.ipynb * Add Bengali Translated Notebooks #Part 12 (OpenMined#3039) Translated Notebooks: Part 12 - Train an Encrypted Neural Network on Encrypted Data.ipynb * fix: failing tutorial translations should be ignored (OpenMined#3047) Exclude only the names, instead of full paths. This means it will effectively exclude translated titles too. * Translation to Portuguese of the tutorials Part 09, 10, and 11 (OpenMined#2980) * add translation to Portuguese of the tutorial Part 02 * (enhance) translate example toy expression literally * (minor fixes) change 'portugues' to 'português' and 'nível superior' to 'parte superior' * add translation to Portuguese of the tutorial Part 03 * (fix) remove old folder name * add translation to Portuguese of the tutorial Part 04 * (update) rename 'portugues' to 'português' and remove old folder * add translation to Portuguese of the tutorial Part 05 * add translation to Portuguese of the tutorial Part 06 * (minor fixes) typo and verb correction * add translation to Portuguese of the tutorial Part 09 * add translation to Portuguese of the tutorial Part 10 * add translation to Portuguese of the tutorial Part 11 * (fix) a cell has been removed by mistake 😑 * (minor fixes) 🤞 🤞 🤞 * (minor fix) full stop added * (fixes) remove repeated paragraphs left in tutorials part 06 and 11 Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: George-Cristian Muraru <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Portuguese translations Part7-8-8bis (OpenMined#2977) * pt translation part07 * middle part 8 translation * fixing portuges * finishing translation part8 and part8 bis * fixing errors * finishing tutorial part 8bis * changing metadata kernel to python3 * update parte07-08 - fixing some errors * Update Parte 07 - Aprendizado Federado com Conjunto de Dados Federado.ipynb * Update Parte 08 - Introdução a Planos.ipynb Co-authored-by: Karl Higley <[email protected]> * Translated tutorial part 01 to Portuguese (OpenMined#3035) Co-authored-by: Karl Higley <[email protected]> * Translate PySyft Tutorials to Portuguese (OpenMined#3020) * Translate notebook Part 13b * translate notebook part 12 bis to portuguese * Update Parte 12 bis - Treinamento criptografado no MNIST.ipynb Co-authored-by: Karl Higley <[email protected]> * Portuguese translation 13c (OpenMined#3023) * translate 13c * translate 13c update * translate 13c update 2 * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb * Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb Co-authored-by: Karl Higley <[email protected]> * No need for requires_grad for input data (OpenMined#2998) * No need for requires_grad for input data * Increase error threshold Co-authored-by: Andrew Trask <[email protected]> * Put the test badges first in README (OpenMined#3046) * Put the test badges first * Update the names of test workflows to clean up README badges Co-authored-by: Théo Ryffel <[email protected]> Co-authored-by: Andrew Trask <[email protected]> * Add Bengali Translated Notebooks #Part 11 (OpenMined#3041) Translated Notebook: Part 11 - Secure Deep Learning Classification.ipynb Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Add instruction for PySyft-Proto dependency and numbered the steps (OpenMined#2818) * Add instruction for PySyft-Proto dependency and numbered the steps * Remove Step 5, it is handled in requirement_dev.txt * Update requirements files to point to syft-proto in PyPI * Remove duplicate dependency (syft-proto) outside requirements.txt * replace proto with syft-proto * Cleanup PATE code (OpenMined#3033) * Format PATE docstrings Uniform indentation First pass of adding type hints Remove commented out TF flags code * Remove commented out print statements in PATE code * Change blank lines in PATE code No blank lines after function doc strings No blank lines immediately after control statements Break up larger chunks of code * Specify l and noise_eps when printing if cant compute sensitivity Before this commit, incorrect noise/moments spammed console with identical statements * Change torch/numpy sensitivity function type hints * Add notice required under Apache 2.0 license that changes have been made to code * Remove ellipses from List in type hints Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * update comment in test units (OpenMined#3018) * remove test unit * Add a missing comma that `black` complains about * add ignore test back and update comment. Co-authored-by: Karl Higley <[email protected]> * Fix potential medium/high severity security holes (OpenMined#3037) * remove security warning messages * Fix missing ast import * Flag the `eval` in `Plan` so that `bandit` security scan ignores it We do want to fix this and find a better way to do it, but don't want it to block adding security scans to, for example, catch future `eval`s added to the code base. * Black formatter wants two spaces before a comment Co-authored-by: Karl Higley <[email protected]> Co-authored-by: Andrew Trask <[email protected]> * CI security testing (OpenMined#3036) * add bandit as dev dependancy * add job for bandit * only report medium and high severity * Match format of security task name with the others Co-authored-by: Karl Higley <[email protected]> * Only test translation notebooks if they have been changed (OpenMined#3060) * Only test translated tutorials that have been changed * Add test_notebooks_basic_translations_diff for GHActions * Change GitHub Actions test for translations * Include git-diff file * Tutorial Notebook 10 note on numpy hook (OpenMined#3022) * added note about notebook not working * phrasing Co-authored-by: Karl Higley <[email protected]> * Revert "Only test translation notebooks if they have been changed (OpenMined#3060)" (OpenMined#3062) This reverts commit 91c8114. * Split tutorial translation tests into separate Github Workflows (OpenMined#3059) * Split tutorial translation tests into separate Github Workflows * Update the main tests workflow to exclude translation tests * Lower-case the directory name for Ukrainian translations * Add update-docker-image.yml to GitHub Actions workflow (OpenMined#3030) * Remove pointer tags/description decode instructions (OpenMined#3067) * Update GH checkout action to v2 (OpenMined#3068) * Fix trigger paths in translation test workflows (OpenMined#3065) * Fix trigger paths in translation test workflows * Update GH checkout action to v2 * Correct build context for dockerhub update via GitHub Actions (OpenMined#3066) * fix build context for dockerhub * Update GH checkout action to v2 Co-authored-by: Karl Higley <[email protected]> * Update msgpack version requirement (OpenMined#3073) * Reduce websocket timeout to one minute (OpenMined#3072) This is making CI tests hang ~forever. I think 278 hours is a bit excessive, don't you? :) * Add wildcards to the tutorial translation test actions (OpenMined#3071) * Translation to Spanish of Parts 09, 13 (a, b and c) (OpenMined#3032) * feat: Translation to Spanish of Parts 09, 13 (a, b and c). Credits to: @thefirebanks This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Fix typos in Spanish tutorials parts 9 and 13b Co-authored-by: Karl Higley <[email protected]> * Translations to Spanish of Tutorials Part 10 and 11 (OpenMined#3031) * feat: Translation to Spanish of Part 10 - Federated Learning with Secure Aggregation. This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * feat: Translation to Spanish of Part 11 - Secure Deep Learning Classification This comes as a response to the initiative to translate all the PySyft tutorials, as it is desirable to learn PySyft in our native language. As requested, every part of the tutorial is done via pull request separately to: - Track better the changes in every document. - Avoid double work. Resolves: OpenMined#2775 See also: OpenMined#2782 * Fix typos in Spanish translations parts 10 and 11 Co-authored-by: Karl Higley <[email protected]> * Notebook 7 hindi translation (OpenMined#3055) * Hindi translations for Part 5, 6, 13b and 13c * Markdown changes applied and github link added * Fixed spaces and translatations * Update Part 06 - Federated Learning on MNIST using a CNN.ipynb * Made changes on translations and spaces * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb * Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb * Updated Part 5 - moving Pygrid Module to PySyft * Hindi translations for Part 4, 7 and 8 updated * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 08 - Introduction to Plans.ipynb * Hindi translations - 4, 7, 8, 8 bis,9 and 10 * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 hindi translation * Update Part 07 - Federated Learning with Federated Dataset.ipynb * Delete Part 07 - Federated Learning with Federated Dataset.ipynb * Notebook 7 - hindi translation Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: Karl Higley <[email protected]> * Bengali 6 (OpenMined#3052) * Bengali 6 Added * Fix markdown link syntax for OpenMined blog post Co-authored-by: Karl Higley <[email protected]> * Bengali 7 (OpenMined#3053) * Bengali 7 Added * Fix link to PySyft project issues Co-authored-by: Karl Higley <[email protected]> * Split the (English) tutorial tests into a separate Github Action (OpenMined#3075) * Split the (English) tutorial tests into a separate Github Action The tutorial tests take about 15 minutes to run, which is approximately equal to the running time of all the rest of the tests. Running them in parallel should speed up the PR checks. * Exclude the isolated file system test fixture from coverage checks It's only used by the notebook tests, which are run separately as of this pull request. * Run notebook tests on all PRs except translation tests Before this change, they would only run on PRs that changed notebooks. * Context of computation for crypten (OpenMined#2963) * New message type for crypten party initialization * Handle CryptenInit message by running local party * Define a new context of computation for crypten The new context of computation can run parties that are distributed across syft workers. The communication of crypten parties remains the same, however, syft workers handle initialization and the serialization of return values. * updated docs * testing crypten context * fix: was setting the bad env variable DISTRIBUTED_BACKEND should have been set and not BACKEND * test serde of CryptenInit message * add crypten as core deps, this should change to extra * delete useless comment * fix: list CryptenInit in OBJ_SIMPLIFIER_AND_DETAILERS * fix msgpack tests * run black * don't cover empty function * Use tag for crypten data (OpenMined#3003) Co-authored-by: Karl Higley <[email protected]> Co-authored-by: arturomf94 <[email protected]> Co-authored-by: Théo Ryffel <[email protected]> Co-authored-by: Arshjot Singh Khehra <[email protected]> Co-authored-by: Vova Manannikov <[email protected]> Co-authored-by: Yugandhar Tripathi <[email protected]> Co-authored-by: Andrew Trask <[email protected]> Co-authored-by: midokura-silvia <[email protected]> Co-authored-by: Carlos Salgado <[email protected]> Co-authored-by: Ricardo Pretelt <[email protected]> Co-authored-by: Sourav Das <[email protected]> Co-authored-by: Aiden V Johnson <[email protected]> Co-authored-by: José Benardi de Souza Nunes <[email protected]> Co-authored-by: Marcus Vinicius <[email protected]> Co-authored-by: systemshift <[email protected]> Co-authored-by: Bogdan Ivanyuk <[email protected]> Co-authored-by: Urvashi Bhattacharyya <[email protected]> Co-authored-by: Jeferson Ferreira <[email protected]> Co-authored-by: João Lucas <[email protected]> Co-authored-by: Márcio Porto <[email protected]> Co-authored-by: Héricles Emanuel <[email protected]> Co-authored-by: Izabella Antonino <[email protected]> Co-authored-by: Bolarinwa Saheed Olayemi <[email protected]> Co-authored-by: Tom <[email protected]> Co-authored-by: froessler <[email protected]> Co-authored-by: Lina Tran <[email protected]> Co-authored-by: Ionésio Junior <[email protected]> Co-authored-by: Henning Dodenhof <[email protected]> Co-authored-by: Sayantan Das <[email protected]> Co-authored-by: Ayoub Benaissa <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Once #3013 is merged to master, we should test the new test unit that was ignored.