Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BIGTOP-3920. Upgrade Hadoop to 3.3.5. #1101

Merged
merged 1 commit into from
Apr 26, 2023
Merged

Conversation

iwasakims
Copy link
Member

https://issues.apache.org/jira/browse/BIGTOP-3920

  • bumped the version and reset the release number.
  • removed 2 patches already contained in Hadoop 3.3.5.

ran smoke-tests of hdfs,yarn,mapreduce on Rocky Linux 8 (x86_64 and aarch64).

While this breaks building HBase and Hive, I will address the issue in follow-up JIRAs (BIGTOP-3926 and BIGTOP-3927).

@guyuqi
Copy link
Member

guyuqi commented Apr 17, 2023

Thanks for working on it. Let me give it a shot on Arm platform, thanks.

@iwasakims
Copy link
Member Author

@guyuqi Are you ok to move this forward for the ease of reviewers of BIGTOP-3927 and BIGTOP-3929? BIGTOP-3930 should be separately addressed since the issue is not related to upgrading Hadoop and other products is affected.

@sekikn
Copy link
Contributor

sekikn commented Apr 26, 2023

+1, tested on master with CentOS 7 and Ubuntu 20.04 using the following commands:

$ ./docker-hadoop.sh -d -C config_centos-7.yaml -L -k hdfs,yarn,mapreduce -s hdfs,yarn,mapreduce -c 3
$ ./docker-hadoop.sh -d -C config_ubuntu-20.04.yaml -L -k hdfs,yarn,mapreduce -s hdfs,yarn,mapreduce -c 3

and on branch-3.2 with Ubuntu 22.04 using the following one:

$ ./docker-hadoop.sh -d -C config_ubuntu-22.04.yaml -F docker-compose-cgroupv2.yml -L -k hdfs,yarn,mapreduce -s hdfs,yarn,mapreduce -c 3

I've only tested it on x86_64, but let's address if there's any error on other platforms. @guyuqi

@sekikn sekikn merged commit 3f90004 into apache:master Apr 26, 2023
sekikn pushed a commit to sekikn/bigtop that referenced this pull request Apr 26, 2023
@sekikn
Copy link
Contributor

sekikn commented Apr 26, 2023

Merged into master and branch-3.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants