Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Translate our tutorials to Spanish #2775

Closed
iamtrask opened this issue Dec 4, 2019 · 5 comments · Fixed by #2785, #2798, #2906, #2920 or #2944
Closed

Translate our tutorials to Spanish #2775

iamtrask opened this issue Dec 4, 2019 · 5 comments · Fixed by #2785, #2798, #2906, #2920 or #2944
Labels
Good first issue 🎓 Perfect for beginners, welcome to OpenMined! Type: New Feature ➕ Introduction of a completely new addition to the codebase

Comments

@iamtrask
Copy link
Member

iamtrask commented Dec 4, 2019

At present, our tutorial notebook series is only available in English, but it would be very desirable for people to be able to learn PySyft in their native language, such as Spanish.

For this PR, copy the current notebooks and translate the inline text to Spanish, placing the notebooks in a "espanol" folder within the "tutorials" folder.

https://github.com/OpenMined/PySyft/tree/master/examples/tutorials

@iamtrask iamtrask added Type: New Feature ➕ Introduction of a completely new addition to the codebase Good first issue 🎓 Perfect for beginners, welcome to OpenMined! labels Dec 4, 2019
@nph4rd
Copy link
Member

nph4rd commented Dec 4, 2019

I'll take this issue :)

@iamtrask
Copy link
Member Author

iamtrask commented Dec 4, 2019

All yours!

iamtrask pushed a commit that referenced this issue Dec 7, 2019
feat: Translation to spanish of the Part 01 - The Basic Tools of 
private deep learning.

This comes as a response to the initiative to translate all the 
PySyft tutorials, as it is desirable to learn PySyft in our native
language.
 
As requested, every part of the tutorial is done via pull request 
separately to:

 - Track better the changes in every document.

 - Avoid double work.
 
Resolves: #2775
See also: #2782
@iamtrask
Copy link
Member Author

iamtrask commented Dec 7, 2019

Heads up @arturomf94 - @ricardopretelt has translated Part 1!

@nph4rd
Copy link
Member

nph4rd commented Dec 7, 2019

Thanks @iamtrask. Indeed, we've been coordinating in order to avoid double work :)

@iamtrask
Copy link
Member Author

iamtrask commented Dec 7, 2019 via email

nph4rd added a commit to nph4rd/PySyft that referenced this issue Dec 9, 2019
feat: Translation to Spanish of the Part 02 - Intro to Federated Learning.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
linamnt pushed a commit to linamnt/PySyft that referenced this issue Dec 9, 2019
feat: Translation to spanish of the Part 01 - The Basic Tools of 
private deep learning.

This comes as a response to the initiative to translate all the 
PySyft tutorials, as it is desirable to learn PySyft in our native
language.
 
As requested, every part of the tutorial is done via pull request 
separately to:

 - Track better the changes in every document.

 - Avoid double work.
 
Resolves: OpenMined#2775
See also: OpenMined#2782
nph4rd added a commit to nph4rd/PySyft that referenced this issue Dec 9, 2019
feat: Translation to Spanish of the Part 02 - Intro to Federated Learning.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
iamtrask pushed a commit that referenced this issue Dec 16, 2019
* Translation to spanish of the tutorial Part 01 

feat: Translation to spanish of the Part 01 - The Basic Tools of 
private deep learning.

This comes as a response to the initiative to translate all the 
PySyft tutorials, as it is desirable to learn PySyft in our native
language.
 
As requested, every part of the tutorial is done via pull request 
separately to:

 - Track better the changes in every document.

 - Avoid double work.
 
Resolves: #2775
See also: #2782

* add translators names in part 1 spanish tutorial

* fix: Add subfolder translations
As requested add all tutorial translations to a single folder
to:

- Work better for the test suit for all the notebooks
- Avoid clutter in the tutorials folder

See also #2782

* Translation to spanish of the tutorial Part 01

fix: Clean the block codes
Some block coded were run, so now are reset so when other users work
the tutorial have a better experience and run them by themselves
iamtrask pushed a commit that referenced this issue Dec 17, 2019
* Translation to Spanish of the tutorial Part 02

feat: Translation to Spanish of the Part 02 - Intro to Federated Learning.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: #2775
See also: #2782

* Translation to Spanish of the tutorial Part 02

feat: Translation to Spanish of the Part 02 - Intro to Federated Learning.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: #2775
See also: #2782

* Fixed typo in contributors

* Add @ricardopretelt as translator

* Translations of tutorials are moved to `examples/tutorials/translations/` to be consistent with tests

See: #2805

* fix: Translation issues in hindi notebook 1 (#2814)

Signed-off-by: Arkadip <[email protected]>

* Remove Siraj Raval mention (#2815)

* Splitnn tutorial (#2813)

* added split neural network tutorial

* Second SplitNN tutorial

* second tutorial

* Third SplitNN tutorial, deleted mnist dataset

* deletion of old notebook

* added images to tutorials

* fixed model shape params

* formatting

* small comments fix

* gradient reference fix

* Fixed installation command in Windows (#2807)

* Improvements to the Federated Recurrent Neural Network notebook

* Removed useless syft imports

* Modified RNN tutorial to also install dependencies from within the notebook

* Added back federated RNN

* Fixed installation instructions

* Translation to Korean of the tutorial Part 01(#2794) (#2804)

* feat: Added the Part 01 Tutorial in Korean (#2794)

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

- Track better the changes in every document.

- Avoid double work.

Resolves: #2794
See also: #2782

* fix: Add subfolder translations
As requested add all tutorial translations to a single folder
to:

- Work better for the test suit for all the notebooks
- Avoid clutter in the tutorials folder

See also #2782

* fix bug with inplace operations on FPT (#2817)

* Improve notebook tests (#2805)

* parameterising Part 12 bis for time

* adding details how to test/contribute notebooks

* move translations in separate folder

* add test for translations and add test for overview of tested notebooks

* fix coverage tests

* fix path issues with import thanks to @arturomf94

* removed too much

* adding SplitNN to tests

* remove print and change last assert

* temp remove tutorial

* update splitnn notebooks for tests

* Translation to Portuguese of the tutorial Part 02 (#2819)

* add translation to Portuguese of the tutorial Part 02

* (enhance) translate example toy expression literally

* Change pysyft_proto to syft_proto (#2820)
iamtrask added a commit that referenced this issue Dec 20, 2019
* Translation to spanish of the tutorial Part 01 

feat: Translation to spanish of the Part 01 - The Basic Tools of 
private deep learning.

This comes as a response to the initiative to translate all the 
PySyft tutorials, as it is desirable to learn PySyft in our native
language.
 
As requested, every part of the tutorial is done via pull request 
separately to:

 - Track better the changes in every document.

 - Avoid double work.
 
Resolves: #2775
See also: #2782

* add translators names in part 1 spanish tutorial

* fix: Add subfolder translations
As requested add all tutorial translations to a single folder
to:

- Work better for the test suit for all the notebooks
- Avoid clutter in the tutorials folder

See also #2782

* Translation to spanish of the tutorial Part 01

fix: Clean the block codes
Some block coded were run, so now are reset so when other users work
the tutorial have a better experience and run them by themselves

* Translation to spanish of the tutorial Part 03
feat: Translation to spanish of the Part 03 - Herramientas avanzadas de ejecución remota
This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.
As requested, every part of the tutorial is done via pull request
separately to:
 - Track better the changes in every document.
 - Avoid double work.

See also: #2782

Co-authored-by: Bobby Wagner <[email protected]>
Co-authored-by: Andrew Trask <[email protected]>
salgadev added a commit to salgadev/PySyft that referenced this issue Jan 14, 2020
feat: Translation to Spanish of tutorial Part 05 - Welcome to the sandbox

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:
- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
karlhigley pushed a commit that referenced this issue Jan 15, 2020
…2906)

* Added Spanish translation for tutorial part 12

* Translation to Spanish of tutorial Part 05

feat: Translation to Spanish of tutorial Part 05 - Welcome to the sandbox

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:
- Track better the changes in every document.
- Avoid double work.

Resolves: #2775
See also: #2782
nph4rd added a commit to nph4rd/PySyft that referenced this issue Jan 17, 2020
* Translation to Spanish of the tutorial Part 04

feat: Translation to Spanish of Part 04 - Federated Learning via Trusted Aggregator.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
karlhigley pushed a commit that referenced this issue Jan 18, 2020
* Translation to Spanish of the tutorial Part 04

feat: Translation to Spanish of Part 04 - Federated Learning via Trusted Aggregator.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: #2775
See also: #2782
nph4rd added a commit to nph4rd/PySyft that referenced this issue Jan 21, 2020
feat: Translation to Spanish of Part 07 - Federated Learning with Federated Dataset.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
karlhigley added a commit that referenced this issue Jan 22, 2020
* Fix typo in translation to Spanish of Tutorial 04

* * Translation to Spanish of the tutorial Part 07

feat: Translation to Spanish of Part 07 - Federated Learning with Federated Dataset.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: #2775
See also: #2782

* Fix typo in translation of Tutorial Part 7

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>
karlhigley added a commit that referenced this issue Jan 22, 2020
* Translation to spanish of the tutorial Part 01 

feat: Translation to spanish of the Part 01 - The Basic Tools of 
private deep learning.

This comes as a response to the initiative to translate all the 
PySyft tutorials, as it is desirable to learn PySyft in our native
language.
 
As requested, every part of the tutorial is done via pull request 
separately to:

 - Track better the changes in every document.

 - Avoid double work.
 
Resolves: #2775
See also: #2782

* add translators names in part 1 spanish tutorial

* fix: Add subfolder translations
As requested add all tutorial translations to a single folder
to:

- Work better for the test suit for all the notebooks
- Avoid clutter in the tutorials folder

See also #2782

* Translation to spanish of the tutorial Part 01

fix: Clean the block codes
Some block coded were run, so now are reset so when other users work
the tutorial have a better experience and run them by themselves

* Translation to spanish of the tutorial Part 03
feat: Translation to spanish of the Part 03 - Herramientas avanzadas de ejecución remota
This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.
As requested, every part of the tutorial is done via pull request
separately to:
 - Track better the changes in every document.
 - Avoid double work.

See also: #2782

* Translation to spanish of the tutorial Part 06

feat: Translation to spanish of the Part 06 - Federated Learning on MNIST using a CNN

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.
As requested, every part of the tutorial is done via pull request
separately to:
 - Track better the changes in every document.
 - Avoid double work.

See also: #2775 and #2782

Co-authored-by: Bobby Wagner <[email protected]>
Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>
salgadev added a commit to salgadev/PySyft that referenced this issue Jan 23, 2020
feat: Translation to spanish of the Part 08 - Introduction to Plans.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:
- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
salgadev added a commit to salgadev/PySyft that referenced this issue Jan 24, 2020
feat: Translation to spanish of the Part 08 - Introduction to Plans.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:
- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
nph4rd added a commit to nph4rd/PySyft that referenced this issue Jan 25, 2020
Changes made by @socd06
This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
iamtrask pushed a commit that referenced this issue Jan 25, 2020
Changes made by @socd06
This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: #2775
See also: #2782
salgadev added a commit to salgadev/PySyft that referenced this issue Jan 30, 2020
feat: Translation to spanish of the Part 12 bis - Encrypted Training on MNIST.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:
- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
salgadev added a commit to salgadev/PySyft that referenced this issue Jan 30, 2020
feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:
- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
salgadev added a commit to salgadev/PySyft that referenced this issue Jan 30, 2020
feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:
- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
karlhigley added a commit that referenced this issue Feb 2, 2020
feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:
- Track better the changes in every document.
- Avoid double work.

Resolves: #2775
See also: #2782

Co-authored-by: Karl Higley <[email protected]>
gmuraru pushed a commit that referenced this issue Feb 5, 2020
This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

Track better the changes in every document.

Avoid double work.

Resolves: #2775
See also: #2782
nph4rd added a commit to nph4rd/PySyft that referenced this issue Feb 7, 2020
Aggregation.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
nph4rd added a commit to nph4rd/PySyft that referenced this issue Feb 7, 2020
…fication

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
nph4rd added a commit to nph4rd/PySyft that referenced this issue Feb 7, 2020
Aggregation.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
nph4rd added a commit to nph4rd/PySyft that referenced this issue Feb 7, 2020
…fication

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
nph4rd added a commit to nph4rd/PySyft that referenced this issue Feb 8, 2020
Credits to: @thefirebanks

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782
karlhigley added a commit that referenced this issue Feb 20, 2020
* feat: Translation to Spanish of Parts 09, 13 (a, b and c).

Credits to: @thefirebanks

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: #2775
See also: #2782

* Fix typos in Spanish tutorials parts 9 and 13b

Co-authored-by: Karl Higley <[email protected]>
karlhigley added a commit that referenced this issue Feb 20, 2020
* feat: Translation to Spanish of Part 10 - Federated Learning with Secure
Aggregation.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: #2775
See also: #2782

* feat: Translation to Spanish of Part 11 - Secure Deep Learning Classification

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

- Track better the changes in every document.
- Avoid double work.

Resolves: #2775
See also: #2782

* Fix typos in Spanish translations parts 10 and 11

Co-authored-by: Karl Higley <[email protected]>
LaRiffle added a commit that referenced this issue Feb 20, 2020
* Migrate CI tests to Github Actions (#2936)

* Create and test pysyft package on various Pythons

Includes Python 2.7, 3.5, 3.6, 3.7

* Rework build script to run tests like Travis

* Update Python versions to 3.6+

* Remove Python 3.8 since Torch doesn't support it

* Only run the tutorial translation tests when the tutorials change

The normal tutorial notebook tests still run on every build, which
should catch the cases where changes to the code break the notebooks.
When the main tutorial notebooks are changed, that will trigger the
action that runs the notebook translation tests, which should surface
the same failures in the whichever translated versions of the tutorials
are affected.

* Measure process time instead of wall time in efficiency tests

This counts only 'user time' and excludes 'system time', and measures
relative to the CPU clock speed, which should make the tests less flaky.

* Increase `max_time` for `test_inv_sym` closer to original value

* Rework "test all notebooks" tests to split off translation tests

* Create a separate section in the action logs for coverage reports

* Exclude translation tests from coverage checks

* Bump `test_inv_sym` execution time limit to 30

* Omit `.github` directory from code coverage checking

* Fix translation notebooks test to account for excluded notebooks

* Move coverage exclusions into setup.cfg instead of .travis.yml

This way they work everywhere that coverage runs, not just on Travis.

* Omit the efficiency tests from Github Action check suite

Omits them from both the tests and the test coverage checks

* Unwind changes to the efficiency tests (now skipped in check suite)

* Stop using efficiency test time assertions outside the efficiency tests

* Standardize the name of the efficiency tests module

* Fix efficiency assertion imports

* Fix a missing wildcard in the coverage omission flag

* Apply testing workflow to PR creation, synchronization, and reopening

* Clean up coverage check inclusions and exclusions

* Fix triggering events section

* feat: Translation to Spanish of Part 08 - Introduction to Plans. (#2962)

Changes made by @socd06
This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: #2775
See also: #2782

* Bump PyTorch to 1.4 (#2930)

Co-authored-by: Théo Ryffel <[email protected]>

* Modify conv implementation and corresponding test to get exact match (#2964)

* Modify conv implementation and corresponding test to get exact match

* Revert mkldnn to the original state after execution

* Exorcise a Python `eval` and transmute into the lesser demon `getattr` (#2951)

* Exorcise a Python `eval` and transmute into the lesser demon `getattr`

* Remove the double split for optimization

* Point to correct version (#2968)

* Refactor Plan internals using PlaceHolder (#2910)

* Add experiment about Plans with placeholders

* Add first implementation of plan using placeholders

* Update notebook of test

* Improve local execution of plans

* Support remote execution of plans

* Rm unused function

* Use Operation object in plan

* Remove fixed precision and sharing from Plan's multiple fetch test

These seem extraneous to the point of the test anyway, so may as well
go ahead and remove them rather than disabling this test.

* Disable encrypted and fixed precision Plans

* Remove unused `plan` attribute on `State`

* Make `test_fetch_plan_multiple_times` pass

* Fix typo in test name

* Update serde_full unit tests (WIP)

* Add example of Plan problem

* Update serde unit tests

* Fix output handling in plans

* Fix nested Plans handling

* Apply formatting to serde helpers

* Adjust serde tests to match the new plan structure

* Update requirements.txt to avoid a merge conflict

* Undo some changes to the Torch hook

* Fix Long vs Float issue in `PointerPlan` test

* Remove current PromiseTensor

* Remove plan.procedure

* Add Trace object to clean code

* Rm promise notebook

* Add support for fix_precision & MPC

* Clean plan code and add a tracer decorator

* Clean state and rm useless functions

* Clean the tracer decorator and comment code

* Add comments to Plan & State new methods

* Fix ordering of arguments in Plan

* Fix dummy error and fix wraps decorators

* Fix

* Add a few comments

* Merge input/output placeholders and do light cleaning

* Rm useless experimental notebook

* Fix serde

* Increase delay in a time test

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: Vova Manannikov <[email protected]>

* add Part 13a Hindi (#2958)

Co-authored-by: Andrew Trask <[email protected]>

* Bump version to `0.2.3.a1` (#2976)

* Refactored code of SplitNN model and added evaluation on datasets (#2983)

* Add Protobuf serialization for Placeholders, Plans, and States (#2972)

* Add Protobuf serialization for Placeholders, Plans, and States

* Bump `syft-proto` requirement to `0.1.1.a1.post5`

Co-authored-by: Andrew Trask <[email protected]>

* Make it possible to JIT trace a PySyft Plan to convert to Torchscript (#2988)

* updated requirements.txt (#2970)

* updated requirements.txt

* Update requirements.txt

Co-authored-by: George-Cristian Muraru <[email protected]>

* Translation to Spanish of Tutorial Part 12 bis (#2979)

feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:
- Track better the changes in every document.
- Avoid double work.

Resolves: #2775
See also: #2782

Co-authored-by: Karl Higley <[email protected]>

* Remove codefix for old pytorch version (#2999)

* feat: Translation to Spanish of part 08 bis (#3011)

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

Track better the changes in every document.

Avoid double work.

Resolves: #2775
See also: #2782

* Added Bengali Translated Notebooks (#2992)

Translated Notebooks:

- Part 13a - Secure Classification with Syft Keras and TFE - Public Training.ipynb
- Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb
- Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Add Udacity Link to Readme (#3016)

* Add sphinx documentation with automatic API building (#3017)

* Add autoapi.extension to sphinx configuration

* Add autoapi as dev dep. Add doc build to travis

* Add doc testenv to github actions and travis

* Fix formatting via black

* Add readthedocs yml configuration

* Configure sphinx to ignore "_static"

* Update theme to sphinx-rtd-theme

* Update index.rst

Co-authored-by: Théo Ryffel <[email protected]>

* Adding some notebooks translated to Portuguese (#3015)

* Adding Part 12 tutorial in portuguese

* Adding Parte13a tutorial in portuguese

Co-authored-by: Andrew Trask <[email protected]>

* trigger github action notebook checks (#3012)

* use doubel qoutes instead

* edit notebook just to trigger testing

* add pull_request trigger condition

* Revert "edit notebook just to trigger testing"

This reverts commit 7e46d5b.

* add path to pull_request

* edit notebook again just to see if checks trigger

* rename notebook .yml file

* edit name string

* change pull_request to master branch condition

* master branch trigger condition

* Revert "edit notebook again just to see if checks trigger"

This reverts commit 60fee72.

* small edit  just to trigger notebook tests, AGAIN!

* Revert "small edit  just to trigger notebook tests, AGAIN!"

This reverts commit 16a83d2.

* Revert changes to `run-tests.yaml`

This one is already working the way we want.

* Make action workflows match

Co-authored-by: Karl Higley <[email protected]>

* Translation into Ukrainian language (#2854)

* Add files via upload

* Update Part 01 - Основні засоби приватного глибинного навчання.ipynb

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: George-Cristian Muraru <[email protected]>
Co-authored-by: Théo Ryffel <[email protected]>

* Remove travis (#3013)

* delete travis.yml

* remove travis from contributing.md

* remoev travis from readme

* add note for future contributers in test_notebooks

* add github actions badge

* edit typo

* add notebook GH test badge

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Hindi translations for Part 5, 6, 13b and 13c (#2909)

* Hindi translations for Part 5, 6, 13b and 13c

* Markdown changes applied and github link added

* Fixed spaces and translatations

* Update Part 06 - Federated Learning on MNIST using a CNN.ipynb

* Made changes on translations and spaces

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Updated Part 5 - moving Pygrid Module to PySyft

* Hindi translations for Part 4, 7 and 8 updated

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 08 - Introduction to Plans.ipynb

* Hindi translations - 4, 7, 8, 8 bis,9 and 10

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 07 - Federated Learning with Federated Dataset.ipynb

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* (minor fix) fix broked badge for run notebook tests (#3028)

* fix: failing tutorial translations should be ignored (#3034)

* Add Bengali Translated Notebooks #Part 12 bis (#3038)

Translated Notebooks:

Part 12 bis - Encrypted Training on MNIST.ipynb

* Add Bengali Translated Notebooks #Part 12 (#3039)

Translated Notebooks:

Part 12 - Train an Encrypted Neural Network on Encrypted Data.ipynb

* fix: failing tutorial translations should be ignored (#3047)

Exclude only the names, instead of full paths. This means it will
effectively exclude translated titles too.

* Translation to Portuguese of the tutorials Part 09, 10, and 11 (#2980)

* add translation to Portuguese of the tutorial Part 02

* (enhance) translate example toy expression literally

* (minor fixes) change 'portugues' to 'português' and 'nível superior' to 'parte superior'

* add translation to Portuguese of the tutorial Part 03

* (fix) remove old folder name

* add translation to Portuguese of the tutorial Part 04

* (update) rename 'portugues' to 'português' and remove old folder

* add translation to Portuguese of the tutorial Part 05

* add translation to Portuguese of the tutorial Part 06

* (minor fixes) typo and verb correction

* add translation to Portuguese of the tutorial Part 09

* add translation to Portuguese of the tutorial Part 10

* add translation to Portuguese of the tutorial Part 11

* (fix) a cell has been removed by mistake 😑

* (minor fixes) 🤞 🤞 🤞

* (minor fix) full stop added

* (fixes) remove repeated paragraphs left in tutorials part 06 and 11

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: George-Cristian Muraru <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Portuguese translations Part7-8-8bis (#2977)

* pt translation part07

* middle part 8 translation

* fixing portuges

* finishing translation part8 and part8 bis

* fixing errors

* finishing tutorial part 8bis

* changing metadata kernel to python3

* update parte07-08 - fixing some errors

* Update Parte 07 - Aprendizado Federado com Conjunto de Dados Federado.ipynb

* Update Parte 08 - Introdução a Planos.ipynb

Co-authored-by: Karl Higley <[email protected]>

* Translated tutorial part 01 to Portuguese (#3035)

Co-authored-by: Karl Higley <[email protected]>

* Translate PySyft Tutorials to Portuguese (#3020)

* Translate notebook Part 13b

* translate notebook part 12 bis to portuguese

* Update Parte 12 bis - Treinamento criptografado no MNIST.ipynb

Co-authored-by: Karl Higley <[email protected]>

* Portuguese translation 13c (#3023)

* translate 13c

* translate 13c update

* translate 13c update 2

* Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb

* Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb

Co-authored-by: Karl Higley <[email protected]>

* No need for requires_grad for input data (#2998)

* No need for requires_grad for input data

* Increase error threshold

Co-authored-by: Andrew Trask <[email protected]>

* Put the test badges first in README (#3046)

* Put the test badges first

* Update the names of test workflows to clean up README badges

Co-authored-by: Théo Ryffel <[email protected]>
Co-authored-by: Andrew Trask <[email protected]>

* Add Bengali Translated Notebooks #Part 11 (#3041)

Translated Notebook:

Part 11 - Secure Deep Learning Classification.ipynb

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Add instruction for PySyft-Proto dependency and numbered the steps (#2818)

* Add instruction for PySyft-Proto dependency and numbered the steps

* Remove Step 5, it is handled in requirement_dev.txt

* Update requirements files to point to syft-proto in PyPI

* Remove duplicate dependency (syft-proto) outside requirements.txt

* replace proto with  syft-proto

* Cleanup PATE code (#3033)

* Format PATE docstrings

Uniform indentation

First pass of adding type hints

Remove commented out TF flags code

* Remove commented out print statements in PATE code

* Change blank lines in PATE code

No blank lines after function doc strings

No blank lines immediately after control statements

Break up larger chunks of code

* Specify l and noise_eps when printing if cant compute sensitivity

Before this commit, incorrect noise/moments spammed console with identical statements

* Change torch/numpy sensitivity function type hints

* Add notice required under Apache 2.0 license that changes have been made to code

* Remove ellipses from List in type hints

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* update comment in test units (#3018)

* remove test unit

* Add a missing comma that `black` complains about

* add ignore test back and update comment.

Co-authored-by: Karl Higley <[email protected]>

* Fix potential medium/high severity security holes (#3037)

* remove security warning messages

* Fix missing ast import

* Flag the `eval` in `Plan` so that `bandit` security scan ignores it

We do want to fix this and find a better way to do it, but don't want it to block adding security scans to, for example, catch future `eval`s added to the code base.

* Black formatter wants two spaces before a comment

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: Andrew Trask <[email protected]>

* CI security testing (#3036)

* add bandit as dev dependancy

* add job for bandit

* only report medium and high severity

* Match format of security task name with the others

Co-authored-by: Karl Higley <[email protected]>

* Only test translation notebooks if they have been changed (#3060)

* Only test translated tutorials that have been changed

* Add test_notebooks_basic_translations_diff for GHActions

* Change GitHub Actions test for translations

* Include git-diff file

* Tutorial Notebook 10 note on numpy hook (#3022)

* added note about notebook not working

* phrasing

Co-authored-by: Karl Higley <[email protected]>

* Revert "Only test translation notebooks if they have been changed (#3060)" (#3062)

This reverts commit 91c8114.

* Split tutorial translation tests into separate Github Workflows (#3059)

* Split tutorial translation tests into separate Github Workflows

* Update the main tests workflow to exclude translation tests

* Lower-case the directory name for Ukrainian translations

* Add update-docker-image.yml to GitHub Actions workflow (#3030)

* Remove pointer tags/description decode instructions (#3067)

* Update GH checkout action to v2 (#3068)

* Fix trigger paths in translation test workflows (#3065)

* Fix trigger paths in translation test workflows

* Update GH checkout action to v2

* Correct build context for dockerhub update via GitHub Actions (#3066)

* fix build context for dockerhub

* Update GH checkout action to v2

Co-authored-by: Karl Higley <[email protected]>

* Update msgpack version requirement (#3073)

* Reduce websocket timeout to one minute (#3072)

This is making CI tests hang ~forever. I think 278 hours is a bit
excessive, don't you? :)

* Add wildcards to the tutorial translation test actions (#3071)

* Translation to Spanish of Parts 09, 13 (a, b and c) (#3032)

* feat: Translation to Spanish of Parts 09, 13 (a, b and c).

Credits to: @thefirebanks

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: #2775
See also: #2782

* Fix typos in Spanish tutorials parts 9 and 13b

Co-authored-by: Karl Higley <[email protected]>

* Translations to Spanish of Tutorials Part 10 and 11 (#3031)

* feat: Translation to Spanish of Part 10 - Federated Learning with Secure
Aggregation.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: #2775
See also: #2782

* feat: Translation to Spanish of Part 11 - Secure Deep Learning Classification

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

- Track better the changes in every document.
- Avoid double work.

Resolves: #2775
See also: #2782

* Fix typos in Spanish translations parts 10 and 11

Co-authored-by: Karl Higley <[email protected]>

* Notebook 7 hindi translation (#3055)

* Hindi translations for Part 5, 6, 13b and 13c

* Markdown changes applied and github link added

* Fixed spaces and translatations

* Update Part 06 - Federated Learning on MNIST using a CNN.ipynb

* Made changes on translations and spaces

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Updated Part 5 - moving Pygrid Module to PySyft

* Hindi translations for Part 4, 7 and 8 updated

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 08 - Introduction to Plans.ipynb

* Hindi translations - 4, 7, 8, 8 bis,9 and 10

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 07 - Federated Learning with Federated Dataset.ipynb

* Notebook 7 hindi translation

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 07 - Federated Learning with Federated Dataset.ipynb

* Notebook 7 - hindi translation

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Bengali 6 (#3052)

* Bengali 6 Added

* Fix markdown link syntax for OpenMined blog post

Co-authored-by: Karl Higley <[email protected]>

* Bengali 7 (#3053)

* Bengali 7 Added

* Fix link to PySyft project issues

Co-authored-by: Karl Higley <[email protected]>

* Split the (English) tutorial tests into a separate Github Action (#3075)

* Split the (English) tutorial tests into a separate Github Action

The tutorial tests take about 15 minutes to run, which is approximately
equal to the running time of all the rest of the tests. Running them in
parallel should speed up the PR checks.

* Exclude the isolated file system test fixture from coverage checks

It's only used by the notebook tests, which are run separately as of
this pull request.

* Run notebook tests on all PRs except translation tests

Before this change, they would only run on PRs that changed notebooks.

* Context of computation for crypten (#2963)

* New message type for crypten party initialization

* Handle CryptenInit message by running local party

* Define a new context of computation for crypten

The new context of computation can run parties that are distributed
across syft workers. The communication of crypten parties remains the
same, however, syft workers handle initialization and the serialization
of return values.

* updated docs

* testing crypten context

* fix: was setting the bad env variable

DISTRIBUTED_BACKEND should have been set and not BACKEND

* test serde of CryptenInit message

* add crypten as core deps, this should change to extra

* delete useless comment

* fix: list CryptenInit in OBJ_SIMPLIFIER_AND_DETAILERS

* fix msgpack tests

* run black

* don't cover empty function

* Use tag for crypten data (#3003)

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: arturomf94 <[email protected]>
Co-authored-by: Théo Ryffel <[email protected]>
Co-authored-by: Arshjot Singh Khehra <[email protected]>
Co-authored-by: Vova Manannikov <[email protected]>
Co-authored-by: Yugandhar Tripathi <[email protected]>
Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: midokura-silvia <[email protected]>
Co-authored-by: Carlos Salgado <[email protected]>
Co-authored-by: Ricardo Pretelt <[email protected]>
Co-authored-by: Sourav Das <[email protected]>
Co-authored-by: Aiden V Johnson <[email protected]>
Co-authored-by: José Benardi de Souza Nunes <[email protected]>
Co-authored-by: Marcus Vinicius <[email protected]>
Co-authored-by: systemshift <[email protected]>
Co-authored-by: Bogdan Ivanyuk <[email protected]>
Co-authored-by: Urvashi Bhattacharyya <[email protected]>
Co-authored-by: Jeferson Ferreira <[email protected]>
Co-authored-by: João Lucas <[email protected]>
Co-authored-by: Márcio Porto <[email protected]>
Co-authored-by: Héricles Emanuel <[email protected]>
Co-authored-by: Izabella Antonino <[email protected]>
Co-authored-by: Bolarinwa Saheed Olayemi <[email protected]>
Co-authored-by: Tom <[email protected]>
Co-authored-by: froessler <[email protected]>
Co-authored-by: Lina Tran <[email protected]>
Co-authored-by: Ionésio Junior <[email protected]>
Co-authored-by: Henning Dodenhof <[email protected]>
Co-authored-by: Sayantan Das <[email protected]>
Co-authored-by: Ayoub Benaissa <[email protected]>
gmuraru added a commit to gmuraru/PySyft that referenced this issue Mar 3, 2020
* Migrate CI tests to Github Actions (OpenMined#2936)

* Create and test pysyft package on various Pythons

Includes Python 2.7, 3.5, 3.6, 3.7

* Rework build script to run tests like Travis

* Update Python versions to 3.6+

* Remove Python 3.8 since Torch doesn't support it

* Only run the tutorial translation tests when the tutorials change

The normal tutorial notebook tests still run on every build, which
should catch the cases where changes to the code break the notebooks.
When the main tutorial notebooks are changed, that will trigger the
action that runs the notebook translation tests, which should surface
the same failures in the whichever translated versions of the tutorials
are affected.

* Measure process time instead of wall time in efficiency tests

This counts only 'user time' and excludes 'system time', and measures
relative to the CPU clock speed, which should make the tests less flaky.

* Increase `max_time` for `test_inv_sym` closer to original value

* Rework "test all notebooks" tests to split off translation tests

* Create a separate section in the action logs for coverage reports

* Exclude translation tests from coverage checks

* Bump `test_inv_sym` execution time limit to 30

* Omit `.github` directory from code coverage checking

* Fix translation notebooks test to account for excluded notebooks

* Move coverage exclusions into setup.cfg instead of .travis.yml

This way they work everywhere that coverage runs, not just on Travis.

* Omit the efficiency tests from Github Action check suite

Omits them from both the tests and the test coverage checks

* Unwind changes to the efficiency tests (now skipped in check suite)

* Stop using efficiency test time assertions outside the efficiency tests

* Standardize the name of the efficiency tests module

* Fix efficiency assertion imports

* Fix a missing wildcard in the coverage omission flag

* Apply testing workflow to PR creation, synchronization, and reopening

* Clean up coverage check inclusions and exclusions

* Fix triggering events section

* feat: Translation to Spanish of Part 08 - Introduction to Plans. (OpenMined#2962)

Changes made by @socd06
This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

* Bump PyTorch to 1.4 (OpenMined#2930)

Co-authored-by: Théo Ryffel <[email protected]>

* Modify conv implementation and corresponding test to get exact match (OpenMined#2964)

* Modify conv implementation and corresponding test to get exact match

* Revert mkldnn to the original state after execution

* Exorcise a Python `eval` and transmute into the lesser demon `getattr` (OpenMined#2951)

* Exorcise a Python `eval` and transmute into the lesser demon `getattr`

* Remove the double split for optimization

* Point to correct version (OpenMined#2968)

* Refactor Plan internals using PlaceHolder (OpenMined#2910)

* Add experiment about Plans with placeholders

* Add first implementation of plan using placeholders

* Update notebook of test

* Improve local execution of plans

* Support remote execution of plans

* Rm unused function

* Use Operation object in plan

* Remove fixed precision and sharing from Plan's multiple fetch test

These seem extraneous to the point of the test anyway, so may as well
go ahead and remove them rather than disabling this test.

* Disable encrypted and fixed precision Plans

* Remove unused `plan` attribute on `State`

* Make `test_fetch_plan_multiple_times` pass

* Fix typo in test name

* Update serde_full unit tests (WIP)

* Add example of Plan problem

* Update serde unit tests

* Fix output handling in plans

* Fix nested Plans handling

* Apply formatting to serde helpers

* Adjust serde tests to match the new plan structure

* Update requirements.txt to avoid a merge conflict

* Undo some changes to the Torch hook

* Fix Long vs Float issue in `PointerPlan` test

* Remove current PromiseTensor

* Remove plan.procedure

* Add Trace object to clean code

* Rm promise notebook

* Add support for fix_precision & MPC

* Clean plan code and add a tracer decorator

* Clean state and rm useless functions

* Clean the tracer decorator and comment code

* Add comments to Plan & State new methods

* Fix ordering of arguments in Plan

* Fix dummy error and fix wraps decorators

* Fix

* Add a few comments

* Merge input/output placeholders and do light cleaning

* Rm useless experimental notebook

* Fix serde

* Increase delay in a time test

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: Vova Manannikov <[email protected]>

* add Part 13a Hindi (OpenMined#2958)

Co-authored-by: Andrew Trask <[email protected]>

* Bump version to `0.2.3.a1` (OpenMined#2976)

* Refactored code of SplitNN model and added evaluation on datasets (OpenMined#2983)

* Add Protobuf serialization for Placeholders, Plans, and States (OpenMined#2972)

* Add Protobuf serialization for Placeholders, Plans, and States

* Bump `syft-proto` requirement to `0.1.1.a1.post5`

Co-authored-by: Andrew Trask <[email protected]>

* Make it possible to JIT trace a PySyft Plan to convert to Torchscript (OpenMined#2988)

* updated requirements.txt (OpenMined#2970)

* updated requirements.txt

* Update requirements.txt

Co-authored-by: George-Cristian Muraru <[email protected]>

* Translation to Spanish of Tutorial Part 12 bis (OpenMined#2979)

feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:
- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

Co-authored-by: Karl Higley <[email protected]>

* Remove codefix for old pytorch version (OpenMined#2999)

* feat: Translation to Spanish of part 08 bis (OpenMined#3011)

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

Track better the changes in every document.

Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

* Added Bengali Translated Notebooks (OpenMined#2992)

Translated Notebooks:

- Part 13a - Secure Classification with Syft Keras and TFE - Public Training.ipynb
- Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb
- Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Add Udacity Link to Readme (OpenMined#3016)

* Add sphinx documentation with automatic API building (OpenMined#3017)

* Add autoapi.extension to sphinx configuration

* Add autoapi as dev dep. Add doc build to travis

* Add doc testenv to github actions and travis

* Fix formatting via black

* Add readthedocs yml configuration

* Configure sphinx to ignore "_static"

* Update theme to sphinx-rtd-theme

* Update index.rst

Co-authored-by: Théo Ryffel <[email protected]>

* Adding some notebooks translated to Portuguese (OpenMined#3015)

* Adding Part 12 tutorial in portuguese

* Adding Parte13a tutorial in portuguese

Co-authored-by: Andrew Trask <[email protected]>

* trigger github action notebook checks (OpenMined#3012)

* use doubel qoutes instead

* edit notebook just to trigger testing

* add pull_request trigger condition

* Revert "edit notebook just to trigger testing"

This reverts commit 7e46d5b.

* add path to pull_request

* edit notebook again just to see if checks trigger

* rename notebook .yml file

* edit name string

* change pull_request to master branch condition

* master branch trigger condition

* Revert "edit notebook again just to see if checks trigger"

This reverts commit 60fee72.

* small edit  just to trigger notebook tests, AGAIN!

* Revert "small edit  just to trigger notebook tests, AGAIN!"

This reverts commit 16a83d2.

* Revert changes to `run-tests.yaml`

This one is already working the way we want.

* Make action workflows match

Co-authored-by: Karl Higley <[email protected]>

* Translation into Ukrainian language (OpenMined#2854)

* Add files via upload

* Update Part 01 - Основні засоби приватного глибинного навчання.ipynb

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: George-Cristian Muraru <[email protected]>
Co-authored-by: Théo Ryffel <[email protected]>

* Remove travis (OpenMined#3013)

* delete travis.yml

* remove travis from contributing.md

* remoev travis from readme

* add note for future contributers in test_notebooks

* add github actions badge

* edit typo

* add notebook GH test badge

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Hindi translations for Part 5, 6, 13b and 13c (OpenMined#2909)

* Hindi translations for Part 5, 6, 13b and 13c

* Markdown changes applied and github link added

* Fixed spaces and translatations

* Update Part 06 - Federated Learning on MNIST using a CNN.ipynb

* Made changes on translations and spaces

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Updated Part 5 - moving Pygrid Module to PySyft

* Hindi translations for Part 4, 7 and 8 updated

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 08 - Introduction to Plans.ipynb

* Hindi translations - 4, 7, 8, 8 bis,9 and 10

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 07 - Federated Learning with Federated Dataset.ipynb

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* (minor fix) fix broked badge for run notebook tests (OpenMined#3028)

* fix: failing tutorial translations should be ignored (OpenMined#3034)

* Add Bengali Translated Notebooks #Part 12 bis (OpenMined#3038)

Translated Notebooks:

Part 12 bis - Encrypted Training on MNIST.ipynb

* Add Bengali Translated Notebooks #Part 12 (OpenMined#3039)

Translated Notebooks:

Part 12 - Train an Encrypted Neural Network on Encrypted Data.ipynb

* fix: failing tutorial translations should be ignored (OpenMined#3047)

Exclude only the names, instead of full paths. This means it will
effectively exclude translated titles too.

* Translation to Portuguese of the tutorials Part 09, 10, and 11 (OpenMined#2980)

* add translation to Portuguese of the tutorial Part 02

* (enhance) translate example toy expression literally

* (minor fixes) change 'portugues' to 'português' and 'nível superior' to 'parte superior'

* add translation to Portuguese of the tutorial Part 03

* (fix) remove old folder name

* add translation to Portuguese of the tutorial Part 04

* (update) rename 'portugues' to 'português' and remove old folder

* add translation to Portuguese of the tutorial Part 05

* add translation to Portuguese of the tutorial Part 06

* (minor fixes) typo and verb correction

* add translation to Portuguese of the tutorial Part 09

* add translation to Portuguese of the tutorial Part 10

* add translation to Portuguese of the tutorial Part 11

* (fix) a cell has been removed by mistake 😑

* (minor fixes) 🤞 🤞 🤞

* (minor fix) full stop added

* (fixes) remove repeated paragraphs left in tutorials part 06 and 11

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: George-Cristian Muraru <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Portuguese translations Part7-8-8bis (OpenMined#2977)

* pt translation part07

* middle part 8 translation

* fixing portuges

* finishing translation part8 and part8 bis

* fixing errors

* finishing tutorial part 8bis

* changing metadata kernel to python3

* update parte07-08 - fixing some errors

* Update Parte 07 - Aprendizado Federado com Conjunto de Dados Federado.ipynb

* Update Parte 08 - Introdução a Planos.ipynb

Co-authored-by: Karl Higley <[email protected]>

* Translated tutorial part 01 to Portuguese (OpenMined#3035)

Co-authored-by: Karl Higley <[email protected]>

* Translate PySyft Tutorials to Portuguese (OpenMined#3020)

* Translate notebook Part 13b

* translate notebook part 12 bis to portuguese

* Update Parte 12 bis - Treinamento criptografado no MNIST.ipynb

Co-authored-by: Karl Higley <[email protected]>

* Portuguese translation 13c (OpenMined#3023)

* translate 13c

* translate 13c update

* translate 13c update 2

* Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb

* Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb

Co-authored-by: Karl Higley <[email protected]>

* No need for requires_grad for input data (OpenMined#2998)

* No need for requires_grad for input data

* Increase error threshold

Co-authored-by: Andrew Trask <[email protected]>

* Put the test badges first in README (OpenMined#3046)

* Put the test badges first

* Update the names of test workflows to clean up README badges

Co-authored-by: Théo Ryffel <[email protected]>
Co-authored-by: Andrew Trask <[email protected]>

* Add Bengali Translated Notebooks #Part 11 (OpenMined#3041)

Translated Notebook:

Part 11 - Secure Deep Learning Classification.ipynb

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Add instruction for PySyft-Proto dependency and numbered the steps (OpenMined#2818)

* Add instruction for PySyft-Proto dependency and numbered the steps

* Remove Step 5, it is handled in requirement_dev.txt

* Update requirements files to point to syft-proto in PyPI

* Remove duplicate dependency (syft-proto) outside requirements.txt

* replace proto with  syft-proto

* Cleanup PATE code (OpenMined#3033)

* Format PATE docstrings

Uniform indentation

First pass of adding type hints

Remove commented out TF flags code

* Remove commented out print statements in PATE code

* Change blank lines in PATE code

No blank lines after function doc strings

No blank lines immediately after control statements

Break up larger chunks of code

* Specify l and noise_eps when printing if cant compute sensitivity

Before this commit, incorrect noise/moments spammed console with identical statements

* Change torch/numpy sensitivity function type hints

* Add notice required under Apache 2.0 license that changes have been made to code

* Remove ellipses from List in type hints

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* update comment in test units (OpenMined#3018)

* remove test unit

* Add a missing comma that `black` complains about

* add ignore test back and update comment.

Co-authored-by: Karl Higley <[email protected]>

* Fix potential medium/high severity security holes (OpenMined#3037)

* remove security warning messages

* Fix missing ast import

* Flag the `eval` in `Plan` so that `bandit` security scan ignores it

We do want to fix this and find a better way to do it, but don't want it to block adding security scans to, for example, catch future `eval`s added to the code base.

* Black formatter wants two spaces before a comment

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: Andrew Trask <[email protected]>

* CI security testing (OpenMined#3036)

* add bandit as dev dependancy

* add job for bandit

* only report medium and high severity

* Match format of security task name with the others

Co-authored-by: Karl Higley <[email protected]>

* Only test translation notebooks if they have been changed (OpenMined#3060)

* Only test translated tutorials that have been changed

* Add test_notebooks_basic_translations_diff for GHActions

* Change GitHub Actions test for translations

* Include git-diff file

* Tutorial Notebook 10 note on numpy hook (OpenMined#3022)

* added note about notebook not working

* phrasing

Co-authored-by: Karl Higley <[email protected]>

* Revert "Only test translation notebooks if they have been changed (OpenMined#3060)" (OpenMined#3062)

This reverts commit 91c8114.

* Split tutorial translation tests into separate Github Workflows (OpenMined#3059)

* Split tutorial translation tests into separate Github Workflows

* Update the main tests workflow to exclude translation tests

* Lower-case the directory name for Ukrainian translations

* Add update-docker-image.yml to GitHub Actions workflow (OpenMined#3030)

* Remove pointer tags/description decode instructions (OpenMined#3067)

* Update GH checkout action to v2 (OpenMined#3068)

* Fix trigger paths in translation test workflows (OpenMined#3065)

* Fix trigger paths in translation test workflows

* Update GH checkout action to v2

* Correct build context for dockerhub update via GitHub Actions (OpenMined#3066)

* fix build context for dockerhub

* Update GH checkout action to v2

Co-authored-by: Karl Higley <[email protected]>

* Update msgpack version requirement (OpenMined#3073)

* Reduce websocket timeout to one minute (OpenMined#3072)

This is making CI tests hang ~forever. I think 278 hours is a bit
excessive, don't you? :)

* Add wildcards to the tutorial translation test actions (OpenMined#3071)

* Translation to Spanish of Parts 09, 13 (a, b and c) (OpenMined#3032)

* feat: Translation to Spanish of Parts 09, 13 (a, b and c).

Credits to: @thefirebanks

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

* Fix typos in Spanish tutorials parts 9 and 13b

Co-authored-by: Karl Higley <[email protected]>

* Translations to Spanish of Tutorials Part 10 and 11 (OpenMined#3031)

* feat: Translation to Spanish of Part 10 - Federated Learning with Secure
Aggregation.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

* feat: Translation to Spanish of Part 11 - Secure Deep Learning Classification

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

* Fix typos in Spanish translations parts 10 and 11

Co-authored-by: Karl Higley <[email protected]>

* Notebook 7 hindi translation (OpenMined#3055)

* Hindi translations for Part 5, 6, 13b and 13c

* Markdown changes applied and github link added

* Fixed spaces and translatations

* Update Part 06 - Federated Learning on MNIST using a CNN.ipynb

* Made changes on translations and spaces

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Updated Part 5 - moving Pygrid Module to PySyft

* Hindi translations for Part 4, 7 and 8 updated

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 08 - Introduction to Plans.ipynb

* Hindi translations - 4, 7, 8, 8 bis,9 and 10

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 07 - Federated Learning with Federated Dataset.ipynb

* Notebook 7 hindi translation

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 07 - Federated Learning with Federated Dataset.ipynb

* Notebook 7 - hindi translation

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Bengali 6 (OpenMined#3052)

* Bengali 6 Added

* Fix markdown link syntax for OpenMined blog post

Co-authored-by: Karl Higley <[email protected]>

* Bengali 7 (OpenMined#3053)

* Bengali 7 Added

* Fix link to PySyft project issues

Co-authored-by: Karl Higley <[email protected]>

* Split the (English) tutorial tests into a separate Github Action (OpenMined#3075)

* Split the (English) tutorial tests into a separate Github Action

The tutorial tests take about 15 minutes to run, which is approximately
equal to the running time of all the rest of the tests. Running them in
parallel should speed up the PR checks.

* Exclude the isolated file system test fixture from coverage checks

It's only used by the notebook tests, which are run separately as of
this pull request.

* Run notebook tests on all PRs except translation tests

Before this change, they would only run on PRs that changed notebooks.

* Context of computation for crypten (OpenMined#2963)

* New message type for crypten party initialization

* Handle CryptenInit message by running local party

* Define a new context of computation for crypten

The new context of computation can run parties that are distributed
across syft workers. The communication of crypten parties remains the
same, however, syft workers handle initialization and the serialization
of return values.

* updated docs

* testing crypten context

* fix: was setting the bad env variable

DISTRIBUTED_BACKEND should have been set and not BACKEND

* test serde of CryptenInit message

* add crypten as core deps, this should change to extra

* delete useless comment

* fix: list CryptenInit in OBJ_SIMPLIFIER_AND_DETAILERS

* fix msgpack tests

* run black

* don't cover empty function

* Use tag for crypten data (OpenMined#3003)

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: arturomf94 <[email protected]>
Co-authored-by: Théo Ryffel <[email protected]>
Co-authored-by: Arshjot Singh Khehra <[email protected]>
Co-authored-by: Vova Manannikov <[email protected]>
Co-authored-by: Yugandhar Tripathi <[email protected]>
Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: midokura-silvia <[email protected]>
Co-authored-by: Carlos Salgado <[email protected]>
Co-authored-by: Ricardo Pretelt <[email protected]>
Co-authored-by: Sourav Das <[email protected]>
Co-authored-by: Aiden V Johnson <[email protected]>
Co-authored-by: José Benardi de Souza Nunes <[email protected]>
Co-authored-by: Marcus Vinicius <[email protected]>
Co-authored-by: systemshift <[email protected]>
Co-authored-by: Bogdan Ivanyuk <[email protected]>
Co-authored-by: Urvashi Bhattacharyya <[email protected]>
Co-authored-by: Jeferson Ferreira <[email protected]>
Co-authored-by: João Lucas <[email protected]>
Co-authored-by: Márcio Porto <[email protected]>
Co-authored-by: Héricles Emanuel <[email protected]>
Co-authored-by: Izabella Antonino <[email protected]>
Co-authored-by: Bolarinwa Saheed Olayemi <[email protected]>
Co-authored-by: Tom <[email protected]>
Co-authored-by: froessler <[email protected]>
Co-authored-by: Lina Tran <[email protected]>
Co-authored-by: Ionésio Junior <[email protected]>
Co-authored-by: Henning Dodenhof <[email protected]>
Co-authored-by: Sayantan Das <[email protected]>
Co-authored-by: Ayoub Benaissa <[email protected]>
gmuraru added a commit to gmuraru/PySyft that referenced this issue Mar 16, 2020
* Migrate CI tests to Github Actions (OpenMined#2936)

* Create and test pysyft package on various Pythons

Includes Python 2.7, 3.5, 3.6, 3.7

* Rework build script to run tests like Travis

* Update Python versions to 3.6+

* Remove Python 3.8 since Torch doesn't support it

* Only run the tutorial translation tests when the tutorials change

The normal tutorial notebook tests still run on every build, which
should catch the cases where changes to the code break the notebooks.
When the main tutorial notebooks are changed, that will trigger the
action that runs the notebook translation tests, which should surface
the same failures in the whichever translated versions of the tutorials
are affected.

* Measure process time instead of wall time in efficiency tests

This counts only 'user time' and excludes 'system time', and measures
relative to the CPU clock speed, which should make the tests less flaky.

* Increase `max_time` for `test_inv_sym` closer to original value

* Rework "test all notebooks" tests to split off translation tests

* Create a separate section in the action logs for coverage reports

* Exclude translation tests from coverage checks

* Bump `test_inv_sym` execution time limit to 30

* Omit `.github` directory from code coverage checking

* Fix translation notebooks test to account for excluded notebooks

* Move coverage exclusions into setup.cfg instead of .travis.yml

This way they work everywhere that coverage runs, not just on Travis.

* Omit the efficiency tests from Github Action check suite

Omits them from both the tests and the test coverage checks

* Unwind changes to the efficiency tests (now skipped in check suite)

* Stop using efficiency test time assertions outside the efficiency tests

* Standardize the name of the efficiency tests module

* Fix efficiency assertion imports

* Fix a missing wildcard in the coverage omission flag

* Apply testing workflow to PR creation, synchronization, and reopening

* Clean up coverage check inclusions and exclusions

* Fix triggering events section

* feat: Translation to Spanish of Part 08 - Introduction to Plans. (OpenMined#2962)

Changes made by @socd06
This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

* Bump PyTorch to 1.4 (OpenMined#2930)

Co-authored-by: Théo Ryffel <[email protected]>

* Modify conv implementation and corresponding test to get exact match (OpenMined#2964)

* Modify conv implementation and corresponding test to get exact match

* Revert mkldnn to the original state after execution

* Exorcise a Python `eval` and transmute into the lesser demon `getattr` (OpenMined#2951)

* Exorcise a Python `eval` and transmute into the lesser demon `getattr`

* Remove the double split for optimization

* Point to correct version (OpenMined#2968)

* Refactor Plan internals using PlaceHolder (OpenMined#2910)

* Add experiment about Plans with placeholders

* Add first implementation of plan using placeholders

* Update notebook of test

* Improve local execution of plans

* Support remote execution of plans

* Rm unused function

* Use Operation object in plan

* Remove fixed precision and sharing from Plan's multiple fetch test

These seem extraneous to the point of the test anyway, so may as well
go ahead and remove them rather than disabling this test.

* Disable encrypted and fixed precision Plans

* Remove unused `plan` attribute on `State`

* Make `test_fetch_plan_multiple_times` pass

* Fix typo in test name

* Update serde_full unit tests (WIP)

* Add example of Plan problem

* Update serde unit tests

* Fix output handling in plans

* Fix nested Plans handling

* Apply formatting to serde helpers

* Adjust serde tests to match the new plan structure

* Update requirements.txt to avoid a merge conflict

* Undo some changes to the Torch hook

* Fix Long vs Float issue in `PointerPlan` test

* Remove current PromiseTensor

* Remove plan.procedure

* Add Trace object to clean code

* Rm promise notebook

* Add support for fix_precision & MPC

* Clean plan code and add a tracer decorator

* Clean state and rm useless functions

* Clean the tracer decorator and comment code

* Add comments to Plan & State new methods

* Fix ordering of arguments in Plan

* Fix dummy error and fix wraps decorators

* Fix

* Add a few comments

* Merge input/output placeholders and do light cleaning

* Rm useless experimental notebook

* Fix serde

* Increase delay in a time test

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: Vova Manannikov <[email protected]>

* add Part 13a Hindi (OpenMined#2958)

Co-authored-by: Andrew Trask <[email protected]>

* Bump version to `0.2.3.a1` (OpenMined#2976)

* Refactored code of SplitNN model and added evaluation on datasets (OpenMined#2983)

* Add Protobuf serialization for Placeholders, Plans, and States (OpenMined#2972)

* Add Protobuf serialization for Placeholders, Plans, and States

* Bump `syft-proto` requirement to `0.1.1.a1.post5`

Co-authored-by: Andrew Trask <[email protected]>

* Make it possible to JIT trace a PySyft Plan to convert to Torchscript (OpenMined#2988)

* updated requirements.txt (OpenMined#2970)

* updated requirements.txt

* Update requirements.txt

Co-authored-by: George-Cristian Muraru <[email protected]>

* Translation to Spanish of Tutorial Part 12 bis (OpenMined#2979)

feat: Translation to Spanish of Tutorial Part 12 bis - Encrypted Training on MNIST.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:
- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

Co-authored-by: Karl Higley <[email protected]>

* Remove codefix for old pytorch version (OpenMined#2999)

* feat: Translation to Spanish of part 08 bis (OpenMined#3011)

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

Track better the changes in every document.

Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

* Added Bengali Translated Notebooks (OpenMined#2992)

Translated Notebooks:

- Part 13a - Secure Classification with Syft Keras and TFE - Public Training.ipynb
- Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb
- Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Add Udacity Link to Readme (OpenMined#3016)

* Add sphinx documentation with automatic API building (OpenMined#3017)

* Add autoapi.extension to sphinx configuration

* Add autoapi as dev dep. Add doc build to travis

* Add doc testenv to github actions and travis

* Fix formatting via black

* Add readthedocs yml configuration

* Configure sphinx to ignore "_static"

* Update theme to sphinx-rtd-theme

* Update index.rst

Co-authored-by: Théo Ryffel <[email protected]>

* Adding some notebooks translated to Portuguese (OpenMined#3015)

* Adding Part 12 tutorial in portuguese

* Adding Parte13a tutorial in portuguese

Co-authored-by: Andrew Trask <[email protected]>

* trigger github action notebook checks (OpenMined#3012)

* use doubel qoutes instead

* edit notebook just to trigger testing

* add pull_request trigger condition

* Revert "edit notebook just to trigger testing"

This reverts commit 7e46d5b.

* add path to pull_request

* edit notebook again just to see if checks trigger

* rename notebook .yml file

* edit name string

* change pull_request to master branch condition

* master branch trigger condition

* Revert "edit notebook again just to see if checks trigger"

This reverts commit 60fee72.

* small edit  just to trigger notebook tests, AGAIN!

* Revert "small edit  just to trigger notebook tests, AGAIN!"

This reverts commit 16a83d2.

* Revert changes to `run-tests.yaml`

This one is already working the way we want.

* Make action workflows match

Co-authored-by: Karl Higley <[email protected]>

* Translation into Ukrainian language (OpenMined#2854)

* Add files via upload

* Update Part 01 - Основні засоби приватного глибинного навчання.ipynb

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: George-Cristian Muraru <[email protected]>
Co-authored-by: Théo Ryffel <[email protected]>

* Remove travis (OpenMined#3013)

* delete travis.yml

* remove travis from contributing.md

* remoev travis from readme

* add note for future contributers in test_notebooks

* add github actions badge

* edit typo

* add notebook GH test badge

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Hindi translations for Part 5, 6, 13b and 13c (OpenMined#2909)

* Hindi translations for Part 5, 6, 13b and 13c

* Markdown changes applied and github link added

* Fixed spaces and translatations

* Update Part 06 - Federated Learning on MNIST using a CNN.ipynb

* Made changes on translations and spaces

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Updated Part 5 - moving Pygrid Module to PySyft

* Hindi translations for Part 4, 7 and 8 updated

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 08 - Introduction to Plans.ipynb

* Hindi translations - 4, 7, 8, 8 bis,9 and 10

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 07 - Federated Learning with Federated Dataset.ipynb

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* (minor fix) fix broked badge for run notebook tests (OpenMined#3028)

* fix: failing tutorial translations should be ignored (OpenMined#3034)

* Add Bengali Translated Notebooks #Part 12 bis (OpenMined#3038)

Translated Notebooks:

Part 12 bis - Encrypted Training on MNIST.ipynb

* Add Bengali Translated Notebooks #Part 12 (OpenMined#3039)

Translated Notebooks:

Part 12 - Train an Encrypted Neural Network on Encrypted Data.ipynb

* fix: failing tutorial translations should be ignored (OpenMined#3047)

Exclude only the names, instead of full paths. This means it will
effectively exclude translated titles too.

* Translation to Portuguese of the tutorials Part 09, 10, and 11 (OpenMined#2980)

* add translation to Portuguese of the tutorial Part 02

* (enhance) translate example toy expression literally

* (minor fixes) change 'portugues' to 'português' and 'nível superior' to 'parte superior'

* add translation to Portuguese of the tutorial Part 03

* (fix) remove old folder name

* add translation to Portuguese of the tutorial Part 04

* (update) rename 'portugues' to 'português' and remove old folder

* add translation to Portuguese of the tutorial Part 05

* add translation to Portuguese of the tutorial Part 06

* (minor fixes) typo and verb correction

* add translation to Portuguese of the tutorial Part 09

* add translation to Portuguese of the tutorial Part 10

* add translation to Portuguese of the tutorial Part 11

* (fix) a cell has been removed by mistake 😑

* (minor fixes) 🤞 🤞 🤞

* (minor fix) full stop added

* (fixes) remove repeated paragraphs left in tutorials part 06 and 11

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: George-Cristian Muraru <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Portuguese translations Part7-8-8bis (OpenMined#2977)

* pt translation part07

* middle part 8 translation

* fixing portuges

* finishing translation part8 and part8 bis

* fixing errors

* finishing tutorial part 8bis

* changing metadata kernel to python3

* update parte07-08 - fixing some errors

* Update Parte 07 - Aprendizado Federado com Conjunto de Dados Federado.ipynb

* Update Parte 08 - Introdução a Planos.ipynb

Co-authored-by: Karl Higley <[email protected]>

* Translated tutorial part 01 to Portuguese (OpenMined#3035)

Co-authored-by: Karl Higley <[email protected]>

* Translate PySyft Tutorials to Portuguese (OpenMined#3020)

* Translate notebook Part 13b

* translate notebook part 12 bis to portuguese

* Update Parte 12 bis - Treinamento criptografado no MNIST.ipynb

Co-authored-by: Karl Higley <[email protected]>

* Portuguese translation 13c (OpenMined#3023)

* translate 13c

* translate 13c update

* translate 13c update 2

* Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb

* Update Parte 13c - Classificação Segura com Syft Keras e TFE - Cliente de predição privada.ipynb

Co-authored-by: Karl Higley <[email protected]>

* No need for requires_grad for input data (OpenMined#2998)

* No need for requires_grad for input data

* Increase error threshold

Co-authored-by: Andrew Trask <[email protected]>

* Put the test badges first in README (OpenMined#3046)

* Put the test badges first

* Update the names of test workflows to clean up README badges

Co-authored-by: Théo Ryffel <[email protected]>
Co-authored-by: Andrew Trask <[email protected]>

* Add Bengali Translated Notebooks #Part 11 (OpenMined#3041)

Translated Notebook:

Part 11 - Secure Deep Learning Classification.ipynb

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Add instruction for PySyft-Proto dependency and numbered the steps (OpenMined#2818)

* Add instruction for PySyft-Proto dependency and numbered the steps

* Remove Step 5, it is handled in requirement_dev.txt

* Update requirements files to point to syft-proto in PyPI

* Remove duplicate dependency (syft-proto) outside requirements.txt

* replace proto with  syft-proto

* Cleanup PATE code (OpenMined#3033)

* Format PATE docstrings

Uniform indentation

First pass of adding type hints

Remove commented out TF flags code

* Remove commented out print statements in PATE code

* Change blank lines in PATE code

No blank lines after function doc strings

No blank lines immediately after control statements

Break up larger chunks of code

* Specify l and noise_eps when printing if cant compute sensitivity

Before this commit, incorrect noise/moments spammed console with identical statements

* Change torch/numpy sensitivity function type hints

* Add notice required under Apache 2.0 license that changes have been made to code

* Remove ellipses from List in type hints

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* update comment in test units (OpenMined#3018)

* remove test unit

* Add a missing comma that `black` complains about

* add ignore test back and update comment.

Co-authored-by: Karl Higley <[email protected]>

* Fix potential medium/high severity security holes (OpenMined#3037)

* remove security warning messages

* Fix missing ast import

* Flag the `eval` in `Plan` so that `bandit` security scan ignores it

We do want to fix this and find a better way to do it, but don't want it to block adding security scans to, for example, catch future `eval`s added to the code base.

* Black formatter wants two spaces before a comment

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: Andrew Trask <[email protected]>

* CI security testing (OpenMined#3036)

* add bandit as dev dependancy

* add job for bandit

* only report medium and high severity

* Match format of security task name with the others

Co-authored-by: Karl Higley <[email protected]>

* Only test translation notebooks if they have been changed (OpenMined#3060)

* Only test translated tutorials that have been changed

* Add test_notebooks_basic_translations_diff for GHActions

* Change GitHub Actions test for translations

* Include git-diff file

* Tutorial Notebook 10 note on numpy hook (OpenMined#3022)

* added note about notebook not working

* phrasing

Co-authored-by: Karl Higley <[email protected]>

* Revert "Only test translation notebooks if they have been changed (OpenMined#3060)" (OpenMined#3062)

This reverts commit 91c8114.

* Split tutorial translation tests into separate Github Workflows (OpenMined#3059)

* Split tutorial translation tests into separate Github Workflows

* Update the main tests workflow to exclude translation tests

* Lower-case the directory name for Ukrainian translations

* Add update-docker-image.yml to GitHub Actions workflow (OpenMined#3030)

* Remove pointer tags/description decode instructions (OpenMined#3067)

* Update GH checkout action to v2 (OpenMined#3068)

* Fix trigger paths in translation test workflows (OpenMined#3065)

* Fix trigger paths in translation test workflows

* Update GH checkout action to v2

* Correct build context for dockerhub update via GitHub Actions (OpenMined#3066)

* fix build context for dockerhub

* Update GH checkout action to v2

Co-authored-by: Karl Higley <[email protected]>

* Update msgpack version requirement (OpenMined#3073)

* Reduce websocket timeout to one minute (OpenMined#3072)

This is making CI tests hang ~forever. I think 278 hours is a bit
excessive, don't you? :)

* Add wildcards to the tutorial translation test actions (OpenMined#3071)

* Translation to Spanish of Parts 09, 13 (a, b and c) (OpenMined#3032)

* feat: Translation to Spanish of Parts 09, 13 (a, b and c).

Credits to: @thefirebanks

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

* Fix typos in Spanish tutorials parts 9 and 13b

Co-authored-by: Karl Higley <[email protected]>

* Translations to Spanish of Tutorials Part 10 and 11 (OpenMined#3031)

* feat: Translation to Spanish of Part 10 - Federated Learning with Secure
Aggregation.

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

 - Track better the changes in every document.

 - Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

* feat: Translation to Spanish of Part 11 - Secure Deep Learning Classification

This comes as a response to the initiative to translate all the
PySyft tutorials, as it is desirable to learn PySyft in our native
language.

As requested, every part of the tutorial is done via pull request
separately to:

- Track better the changes in every document.
- Avoid double work.

Resolves: OpenMined#2775
See also: OpenMined#2782

* Fix typos in Spanish translations parts 10 and 11

Co-authored-by: Karl Higley <[email protected]>

* Notebook 7 hindi translation (OpenMined#3055)

* Hindi translations for Part 5, 6, 13b and 13c

* Markdown changes applied and github link added

* Fixed spaces and translatations

* Update Part 06 - Federated Learning on MNIST using a CNN.ipynb

* Made changes on translations and spaces

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Update Part 13c - Secure Classification with Syft Keras and TFE - Private Prediction Client.ipynb

* Update Part 13b - Secure Classification with Syft Keras and TFE - Secure Model Serving.ipynb

* Updated Part 5 - moving Pygrid Module to PySyft

* Hindi translations for Part 4, 7 and 8 updated

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 08 - Introduction to Plans.ipynb

* Hindi translations - 4, 7, 8, 8 bis,9 and 10

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 07 - Federated Learning with Federated Dataset.ipynb

* Notebook 7 hindi translation

* Update Part 07 - Federated Learning with Federated Dataset.ipynb

* Delete Part 07 - Federated Learning with Federated Dataset.ipynb

* Notebook 7 - hindi translation

Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: Karl Higley <[email protected]>

* Bengali 6 (OpenMined#3052)

* Bengali 6 Added

* Fix markdown link syntax for OpenMined blog post

Co-authored-by: Karl Higley <[email protected]>

* Bengali 7 (OpenMined#3053)

* Bengali 7 Added

* Fix link to PySyft project issues

Co-authored-by: Karl Higley <[email protected]>

* Split the (English) tutorial tests into a separate Github Action (OpenMined#3075)

* Split the (English) tutorial tests into a separate Github Action

The tutorial tests take about 15 minutes to run, which is approximately
equal to the running time of all the rest of the tests. Running them in
parallel should speed up the PR checks.

* Exclude the isolated file system test fixture from coverage checks

It's only used by the notebook tests, which are run separately as of
this pull request.

* Run notebook tests on all PRs except translation tests

Before this change, they would only run on PRs that changed notebooks.

* Context of computation for crypten (OpenMined#2963)

* New message type for crypten party initialization

* Handle CryptenInit message by running local party

* Define a new context of computation for crypten

The new context of computation can run parties that are distributed
across syft workers. The communication of crypten parties remains the
same, however, syft workers handle initialization and the serialization
of return values.

* updated docs

* testing crypten context

* fix: was setting the bad env variable

DISTRIBUTED_BACKEND should have been set and not BACKEND

* test serde of CryptenInit message

* add crypten as core deps, this should change to extra

* delete useless comment

* fix: list CryptenInit in OBJ_SIMPLIFIER_AND_DETAILERS

* fix msgpack tests

* run black

* don't cover empty function

* Use tag for crypten data (OpenMined#3003)

Co-authored-by: Karl Higley <[email protected]>
Co-authored-by: arturomf94 <[email protected]>
Co-authored-by: Théo Ryffel <[email protected]>
Co-authored-by: Arshjot Singh Khehra <[email protected]>
Co-authored-by: Vova Manannikov <[email protected]>
Co-authored-by: Yugandhar Tripathi <[email protected]>
Co-authored-by: Andrew Trask <[email protected]>
Co-authored-by: midokura-silvia <[email protected]>
Co-authored-by: Carlos Salgado <[email protected]>
Co-authored-by: Ricardo Pretelt <[email protected]>
Co-authored-by: Sourav Das <[email protected]>
Co-authored-by: Aiden V Johnson <[email protected]>
Co-authored-by: José Benardi de Souza Nunes <[email protected]>
Co-authored-by: Marcus Vinicius <[email protected]>
Co-authored-by: systemshift <[email protected]>
Co-authored-by: Bogdan Ivanyuk <[email protected]>
Co-authored-by: Urvashi Bhattacharyya <[email protected]>
Co-authored-by: Jeferson Ferreira <[email protected]>
Co-authored-by: João Lucas <[email protected]>
Co-authored-by: Márcio Porto <[email protected]>
Co-authored-by: Héricles Emanuel <[email protected]>
Co-authored-by: Izabella Antonino <[email protected]>
Co-authored-by: Bolarinwa Saheed Olayemi <[email protected]>
Co-authored-by: Tom <[email protected]>
Co-authored-by: froessler <[email protected]>
Co-authored-by: Lina Tran <[email protected]>
Co-authored-by: Ionésio Junior <[email protected]>
Co-authored-by: Henning Dodenhof <[email protected]>
Co-authored-by: Sayantan Das <[email protected]>
Co-authored-by: Ayoub Benaissa <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment