Skip to content
This repository was archived by the owner on Jun 2, 2024. It is now read-only.

Proposing some useful defaults #8

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Proposing some useful defaults #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
172deee
Adding some useful defaults
Jan 23, 2020
c554d02
higher defaults
saif-ellafi Feb 2, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions cluster-setup/local-cluster/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,24 @@ services:
links:
- hadoop-namenode
tty: true
hadoop-datanode2:
image: segence/hadoop
container_name: hadoop-datanode2
hostname: hadoop-datanode2
ports:
- "8043:8043"
- "50076:50076"
expose:
- "34000-48000"
- "50010"
- "50020"
- "50040"
environment:
- HDFS_REPLICATION_FACTOR=1
- HADOOP_NAMENODE_HOST=hadoop-namenode
volumes:
- ./data-slave2:/data
- ./slaves-config:/config:ro
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ro addition is good, I guess we can add it to slave1 as well?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does have it already

links:
- hadoop-namenode
tty: true
1 change: 1 addition & 0 deletions cluster-setup/local-cluster/slaves-config/slaves
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
hadoop-datanode1
hadoop-datanode2
12 changes: 12 additions & 0 deletions docker-container/config/yarn-site.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,4 +41,16 @@
<name>yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage</name>
<value>98.5</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>4096</value>
</property>
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>2.1</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-vcores</name>
<value>4</value>
</property>
</configuration>