Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix parameter problems #2

Merged
merged 3 commits into from
Dec 7, 2018
Merged

Fix parameter problems #2

merged 3 commits into from
Dec 7, 2018

Conversation

harmony-ek
Copy link
Contributor

This fixes the libRaptorQ zero-as-error return issue. Essentially, enc->compute(…) needed to be called in advance to prime the matrices, as the RFC version of the API does not create the per-source-block encoder on demand.

This patch also converts zero value returned by enc->encode(…) into a proper error, and properly handles explicitly-sized integer conversions between C++ and Go.

Eugene Kim added 3 commits December 6, 2018 23:53
This required reimplementing HostToNetXX/NetToHostXX functions as
helpers instead of simply exposing templated version under custom names,
because apparently typemap intercept does not work on template argument
types.
This is not done automatically, but is required when using the RFC
version of libRaptorQ.
@harmony-ek harmony-ek self-assigned this Dec 7, 2018
Copy link

@LeoHChen LeoHChen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add test logs and how did you test the code.

@@ -1,12 +1,26 @@
%{
#include <stdint.h>
#include <cstdint>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file auto-generated by swig?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. stdint.swg is our own utility SWIG module that establishes mapping between
C++ uintXX_t/intXX_t types and Go uintXX/intXX types.

@@ -728,23 +728,27 @@ bool set_compression(Compress const compression);
size_t local_cache_size (size_t const local_cache);
size_t get_local_cache_size();

namespace Impl {
namespace Endian {
} // namespace RaptorQ__v1
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this file generated by swig?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, nope. This file is a SWIG source file – a (partial) interface definition of the wrapped C++ code to be converted into target (Go) language binding.

@@ -12,7 +12,7 @@ type EncoderFactory struct {

func (*EncoderFactory) New(input []byte, symbolSize uint16, minSubSymbolSize uint16,
maxSubBlockSize uint32, alignment uint8) (enc raptorq.Encoder, err error) {
wrapped := swig.NewBytesEncoder(input, minSubSymbolSize, symbolSize,
wrapped := swig.InitBytesEncoder(input, minSubSymbolSize, symbolSize,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any comment here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

InitBytesEncoder = NewBytesEncoder (BytesEncoder ctor) call + its compute(…) method call. See the inlined definition in swig.swigcxx for details.

@@ -1094,3 +1094,20 @@ public:

%template(BytesEncoder) RFC6330__v1::Impl::Encoder<unsigned char *, unsigned char *>;
%template(BytesDecoder) RFC6330__v1::Impl::Decoder<unsigned char *, unsigned char *>;

%inline %{
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it generated by swig?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above. FWIW, none of these files have been generated by SWIG. They are fed into SWIG though. SWIG-generated files are used internally when we run go build on this package.

@harmony-ek
Copy link
Contributor Author

Tested using @chaosma's unicast IDA code. The test file could not be transferred without the fix because the encoder refused to generate any symbols; with the fix the encoder generates symbols, the IDA client sends the generated symbols to the IDA server, which then decodes the symbols and recovers the original file.

quelthalas 23:46:27 ida (master) $ 709 ls -l
total 40
-rw-r--r--  1 ek  staff   1323 Nov 29 13:15 main.go
drwxr-xr-x  6 ek  staff    192 Dec  6 23:45 raptorq
-rw-r--r--  1 ek  staff    290 Nov 29 13:13 readme.txt
drwxr-xr-x  4 ek  staff    128 Dec  6 23:46 received
-rw-r--r--  1 ek  staff  10329 Nov 29 13:13 test.txt
quelthalas 23:46:39 ida (master) $ 710 
quelthalas 23:46:42 ida (master) $ 711 ls -l received/
total 48
-rw-r--r--  1 ek  staff  10329 Dec  6 23:40 2625048570362569127
-rw-r--r--  1 ek  staff  10329 Dec  6 23:46 4962440957302961228
quelthalas 23:46:51 ida (master) $ 713 openssl sha256 received/* test.txt 
SHA256(received/2625048570362569127)= 07c7c566bf2cf9db41de68b6f53cc81db737a9c78b6246bae6beb004b70010a8
SHA256(received/4962440957302961228)= 07c7c566bf2cf9db41de68b6f53cc81db737a9c78b6246bae6beb004b70010a8
SHA256(test.txt)= 07c7c566bf2cf9db41de68b6f53cc81db737a9c78b6246bae6beb004b70010a8

@LeoHChen
Copy link

LeoHChen commented Dec 7, 2018

LGTM.

@LeoHChen
Copy link

LeoHChen commented Dec 7, 2018

How come I can't approve the PR on github?

Copy link

@chaosma chaosma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works on my IDA broadcast code

@harmony-ek harmony-ek merged commit 9c191c6 into master Dec 7, 2018
@harmony-ek harmony-ek deleted the fix_parameter_problems branch December 7, 2018 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants