-
Notifications
You must be signed in to change notification settings - Fork 179
NCStream endianness incorrect #442
Comments
attn @JohnLCaron |
And for reference, the first 32 bytes across the wire for an int64 variable with [0, 1, 2, 3] is:
|
It looks like this only applies to simple data variables. The |
Im wondering if this is the problem: public DataOutputStream setupStream(OutputStream out, int size)
throws IOException
{
switch (type) {
// For compression (currently deflate) we compress the data, then
// will write the block size, and then data, when the stream is closed.
case DEFLATE:
// limit level to range [-1, 9], where -1 is default deflate setting.
int level = Math.min(Math.max((Integer)compressInfo, -1), 9);
int bufferSize = Math.min(size / 2, 512 * 1024 * 1024);
return new NcStreamCompressedOutputStream(out, bufferSize, level);
default:
System.out.printf(" Unknown compression type %s. Defaulting to none.%n", type);
// In the case of no compression, go ahead and write the block
// size so that the stream is ready for data
case NONE:
DataOutputStream dos = new DataOutputStream(out);
NcStream.writeVInt(dos, size);
return dos;
}
} should be public OutputStream setupStream(OutputStream out, int size)
throws IOException
{
switch (type) {
// For compression (currently deflate) we compress the data, then
// will write the block size, and then data, when the stream is closed.
case DEFLATE:
// limit level to range [-1, 9], where -1 is default deflate setting.
int level = Math.min(Math.max((Integer)compressInfo, -1), 9);
int bufferSize = Math.min(size / 2, 512 * 1024 * 1024);
return new NcStreamCompressedOutputStream(out, bufferSize, level);
default:
System.out.printf(" Unknown compression type %s. Defaulting to none.%n", type);
// In the case of no compression, go ahead and write the block
// size so that the stream is ready for data
case NONE:
NcStream.writeVInt(out, size);
return out;
}
} and NcStreamCompressedOutputStream extends OutputStream ?? |
Close--and you had me believing it. In fact, I was able to eliminate the use of public static long copyToOutputStream(Array data, OutputStream out) throws java.io.IOException {
Class classType = data.getElementType();
DataOutputStream dataOut;
if (out instanceof DataOutputStream)
dataOut = (DataOutputStream) out;
else
dataOut = new DataOutputStream(out);
... |
yep, but weird cdmremote has worked up to now. On Wed, Feb 24, 2016 at 1:21 PM, Ryan May [email protected] wrote:
|
Could it be using |
well, at least i was right that we needed another implementation to tease On Wed, Feb 24, 2016 at 2:09 PM, Ryan May [email protected] wrote:
|
Did this get fixed? |
No, it wasn't clear to me whether modifying |
I don't think we're encoding endianness correctly on ncstream. (For the following, all lines are taken from 4.6). For starters, we say (NcStreamWriter.java:105):
and when we write the protos the wire this becomes (NcStream.java:225):
So running on my machine, NcStream sends
bigend: false
(in the protobuf) across the wire.However, writing the data ends up going down (for a long int) to (DataOutputStream.java:215, a jdk class):
That's not native encoding; it's also not little endian, which is what the
bigend: false
would have me believe. So, do we fix the code at least to properly encodebigend: true
(which would always be true, at least for the current java implementation), or do we redefine the spec to always be big endian, and say to ignore the flag?The text was updated successfully, but these errors were encountered: