-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"TypeError: a bytes-like object is required, not 'str'" with HDFS/Hadoop #2402
Comments
Smaller code example, same problem:
|
It turns out this has nothing to do with HdfsTarget, I can execute the task without problems when calling it from the commandline as "PYTHONPATH=. luigi ". |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I saw several issues mentioning the error message "TypeError: a bytes-like object is required, not 'str'".
From the other issues, I could not derive how I can resolve my problem.
For me, the error message occurs when reading and writing HDFS targets:
Consequently, one of my tasks fails:
My code parses a sequence of characters, e.g., "AAA", and dynamically chains three "ATask" MapReduce jobs (later, I want to be able to produce chains of different jobs).
It all works fine when I read and write local targets, the jobs get chained and use intermediate temporary files "temp0.json", "temp1.json", ... to exchange data between tasks.
But when reading and writing HDFS targets, I get the above error message.
This is my code:
My textual input is of the form
The text was updated successfully, but these errors were encountered: