내용

글번호 719
작성자 허진경
작성일 2017-07-17 12:20:29
제목 하둡 클러스터 구축 실습 1/4 - 설정파일 수정
내용 login as: hadoop hadoop@192.168.75.128's password: Last login: Fri Jul 14 00:08:17 2017 from 192.168.75.1 [hadoop@master ~]$ ssh slave1 Last login: Fri Jul 14 01:20:59 2017 from master [hadoop@slave1 ~]$ exit logout Connection to slave1 closed. [hadoop@master ~]$ ssh slave2 Last login: Fri Jul 14 01:21:55 2017 from master [hadoop@slave2 ~]$ exit logout Connection to slave2 closed. [hadoop@master ~]$ ssh slave3 Last login: Fri Jul 14 01:22:22 2017 from master [hadoop@slave3 ~]$ exit logout Connection to slave3 closed. [hadoop@master ~]$ cd hadoop-2.8.0/etc/hadoop/ [hadoop@master hadoop]$ vi hadoop-env.sh [hadoop@master hadoop]$ vi core-site.xml [hadoop@master hadoop]$ cat core-site.xml <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://master:9000</value> </property> </configuration> [hadoop@master hadoop]$ vi hdfs-site.xml [hadoop@master hadoop]$ cat hdfs-site.xml <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>file:///home/hadoop/dfs/name</value> </property> <property> <name>dfs.namenode.edits.dir</name> <value>file:///home/hadoop/dfs/edits</value> </property> <property> <name>dfs.namenode.secondary.http-address</name> <value>slave3:50090</value> </property> </configuration> [hadoop@master hadoop]$ cp mapred-site.xml.template mapred-site.xml [hadoop@master hadoop]$ vi mapred-site.xml [hadoop@master hadoop]$ cat mapred-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration> [hadoop@master hadoop]$ [hadoop@master hadoop]$ vi yarn-site.xml <?xml version="1.0"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <configuration> <!-- Site specific YARN configuration properties --> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> </configuration> [hadoop@master hadoop]$ vi slaves [hadoop@master hadoop]$ cat slaves slave1 slave2 slave3 [hadoop@master hadoop]$ ssh slave1 Last login: Sun Jul 16 19:42:46 2017 from master [hadoop@slave1 ~]$ cd hadoop-2.8.0/etc/hadoop/ [hadoop@slave1 hadoop]$ vi core-site.xml [hadoop@slave1 hadoop]$ cat core-site.xml <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://master:9000</value> </property> </configuration> [hadoop@slave1 hadoop]$ vi hdfs-site.xml [hadoop@slave1 hadoop]$ cat hdfs-site.xml <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>dfs.datanode.data.dir</name> <value>file:///home/hadoop/dfs/data</value> </property> </configuration> [hadoop@slave1 hadoop]$ rsync -av hdfs-site.xml slave2:~/hadoop-2.8.0/etc/hadoop/ The authenticity of host 'slave2 (192.168.75.195)' can't be established. ECDSA key fingerprint is 1f:9d:29:36:7c:3e:8d:9b:74:4a:92:f3:1e:9d:93:a3. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'slave2,192.168.75.195' (ECDSA) to the list of known hosts. hadoop@slave2's password: sending incremental file list hdfs-site.xml sent 290 bytes received 43 bytes 74.00 bytes/sec total size is 888 speedup is 2.67 [hadoop@slave1 hadoop]$ rsync -av hdfs-site.xml slave3:~/hadoop-2.8.0/etc/hadoop/ The authenticity of host 'slave3 (192.168.75.177)' can't be established. ECDSA key fingerprint is 1f:9d:29:36:7c:3e:8d:9b:74:4a:92:f3:1e:9d:93:a3. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'slave3,192.168.75.177' (ECDSA) to the list of known hosts. hadoop@slave3's password: sending incremental file list hdfs-site.xml sent 290 bytes received 43 bytes 95.14 bytes/sec total size is 888 speedup is 2.67 [hadoop@slave1 hadoop]$ [hadoop@slave1 hadoop]$ exit logout Connection to slave1 closed. [hadoop@master hadoop]$ ssh slave3 Last login: Sun Jul 16 17:42:58 2017 from master [hadoop@slave3 ~]$ cd hadoop-2.8.0/etc/hadoop/ [hadoop@slave3 hadoop]$ vi hdfs-site.xml [hadoop@slave3 hadoop]$ cat hdfs-site.xml <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>dfs.datanode.data.dir</name> <value>file:///home/hadoop/dfs/data</value> </property> <property> <name>dfs.namenode.checkpoint.dir</name> <value>file:///home/hadoop/dfs/namesecondary</value> </property> </configuration> [hadoop@slave3 hadoop]$ [hadoop@slave3 hadoop]$ exit logout Connection to slave3 closed. [hadoop@master hadoop]$