============ 软件的安装 ============ 安装 Docker ================= To build the *Docker* container, Docker need to be installed: * please refer to `Docker Installation Homepage `_ * quick installation .. code-block:: bash $ sudo apt-get update $ sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - $ sudo apt-key fingerprint 0EBFCD88 $ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable" $ sudo apt-get update $ sudo apt-get install docker-ce docker-ce-cli containerd.io .. note:: if you have following error message: .. code-block:: bash $ docker images Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.40/images/json: dial unix /var/run/docker.sock: connect: permission denied please do: .. code-block:: bash $ sudo groupadd docker $ sudo usermod -aG docker ${USER} $ You would need to loog out and log back (looks like reboot needed) in so that your group membership is re-evaluated .. note:: if you get following error message: .. code-block:: bash $ docker container ls -a Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? please do: .. code-block:: bash $ sudo service docker start * check if the docker installation is good .. code-block:: bash $ docker version Client: Docker Engine - Community Version: 19.03.3 API version: 1.40 Go version: go1.12.10 Git commit: a872fc2f86 Built: Tue Oct 8 00:58:10 2019 OS/Arch: linux/amd64 Experimental: false Server: Docker Engine - Community Engine: Version: 19.03.3 API version: 1.40 (minimum version 1.12) Go version: go1.12.10 Git commit: a872fc2f86 Built: Tue Oct 8 00:56:46 2019 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.2.10 GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339 runc: Version: 1.0.0-rc8+dev GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657 docker-init: Version: 0.18.0 GitCommit: fec3683 安装 Docker Compose ========================== To build the *Docker Compose*: * please refer to `Docker compose Homepage `_ * quick installation (Please check the latest stable version number) .. code-block:: bash $ sudo curl -L "https://github.com/docker/compose/releases/download/1.25.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose $ sudo chmod +x /usr/local/bin/docker-compose $ docker-compose --version docker-compose version 1.24.1, build 4667896b 安装 Charliecloud ========================= The whole system requires to be run with *Charliecloud*. To install Charliecloud: * please refer to `Charliecloud Homepage `_ * quick installation Use Spack to install as a example ( Please refer to `Spack package management `_ ) .. code-block:: bash $ spack install pkg-config # This is bug fix $ spack install charliecloud + builder * check if charliecloud installation is OK .. code-block:: bash $ ch-run --version 0.12~pre+59b5bfd 安装 Spack ========================= 了解详情，`请点击这里 `_ .. code-block:: bash $ git clone https://github.com/spack/spack.git # Add spack/bin to "PATH" $ export PATH=/path/of/spack/bin:$PATH $ . /path/of/spack/share/spack/setup-env.sh $ spack --version 0.14.2-1486-b80d5e7 制作 Spack 镜像 ================= 如果本机（工作机器）无法连接互联网，需要找一台可以连入互联网的计算机，同样安装 `Spack `_ ，该台计算机将用于制作 Spack 的镜像软件仓库。 .. note:: 两台计算机必须安装同样版本的 Spack！ 在可以连入互联网的计算机上也安装好了 Spack 之后，开始在该计算机上下载安装 `METplus `_ 所需要的软件。 1. 建立一个 :code:`specs.txt` 文件，列出所需软件： .. code-block:: bash gcc@9.2.0 openmpi@3.1.6 netcdf-fortran@4.5.2%gcc@9.2.0 ^netcdf-c@4.7.3+parallel-netcdf ^hdf5@1.10.6+fortran py-matplotlib@3.2.2 py-cartopy@0.17.0 #. 制作需要软件的镜像 .. code-block:: bash $ spack mirror create -D -f specs.txt 制作完成后，所需软件包下载到如下目录： .. code-block:: bash $ home/xinzhang> tree -L 1 /glade/u/home/xinzhang/spack/var/spack/cache /glade/u/home/xinzhang/spack/var/spack/cache ├── autoconf ├── automake ├── gdbm ├── hdf5 ├── hwloc ├── libiconv ├── libpciaccess ├── libsigsegv ├── libtool ├── libxml2 ├── m4 ├── ncurses ├── netcdf-c ├── netcdf-fortran ├── numactl ├── openmpi ├── parallel-netcdf ├── perl ├── pkgconf ├── readline ├── _source-cache ├── util-macros ├── xz └── zlib 24 directories, 0 files #. 打包镜像目录，上传到需要安装软件的工作机器： .. code-block:: bash $ cd /glade/u/home/xinzhang/spack/var/spack $ tar -zcvf spack_mirror.tgz cache #. 在工作机器上，设置 Spack 镜像目录 在工作机器上，将 :code:`spack_mirror.tgz` 文件解压到一个目录，如： .. code-block:: bash $ mkdir -p /home/wrf/spack_mirror $ cd /home/wrf/spack_mirror $ tar xzvf ~/spack_mirror.tgz $ rm ~/spack_mirror.tgz 设置 Spack 镜像目录 .. code-block:: bash $ spack mirror add local_filesytem file:///home/wrf/spack_mirror/cache $ spack mirror list local_filesyetm file:///home/wrf/home/wrf/spack_mirror/cache 相关软件安装 ================= 1. 安装编译器 gnu@9.2.0 - 设置当前编译器 .. code-block:: bash $ spack compiler find ==> Added 1 new compiler to /home/wrf/.spack/linux/compilers.yaml gcc@4.8.5 ==> Compilers are defined in the following files: /home/wrf/.spack/linux/compilers.yaml $ cat /home/wrf/.spack/linux/compilers.yaml compilers: - compiler: spec: gcc@4.8.5 paths: cc: /usr/bin/gcc cxx: /usr/bin/g++ f77: /usr/bin/gfortran fc: /usr/bin/gfortran flags: {} operating_system: centos7 target: x86_64 modules: [] environment: unset: [] extra_rpaths: [] - 安装 gcc@9.2.0 .. code-block:: bash $ spack install gcc@9.2.0 $ spack find ==> 16 installed packages -- linux-centos7-haswell / gcc@4.8.5 ---------------------------- autoconf@2.69 gcc@9.2.0 gmp@6.1.2 libsigsegv@2.12 m4@1.4.18 mpfr@3.1.6 perl@5.30.2 readline@8.0 automake@1.16.2 gdbm@1.18.1 isl@0.20 libtool@2.4.6 mpc@1.1.0 ncurses@6.2 pkgconf@1.7.3 zlib@1.2.11 - 装载 gcc@9.2.0 .. code-block:: bash $ spack load gcc@9.2.0 $ which gfortran ~/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-9.2.0-tl6spk7vtzi2ywa3b2ev2hecrcrfuvuc/bin/gfortran $ gfortran --version GNU Fortran (Spack GCC) 9.2.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. - 将 gcc@9.2.0 加入 Spack .. code-block:: bash $ spack compiler find ==> Added 1 new compiler to /home/wrf/.spack/linux/compilers.yaml gcc@9.2.0 ==> Compilers are defined in the following files: /home/wrf/.spack/linux/compilers.yaml #. 安装 netcdf .. code-block:: bash $ spack install netcdf-fortran@4.5.2%gcc@9.2.0 ^netcdf-c@4.7.3+parallel-netcdf ^hdf5@1.10.6+fortran $ spack find ==> 39 installed packages -- linux-centos7-haswell / gcc@4.8.5 ---------------------------- autoconf@2.69 gcc@9.2.0 gmp@6.1.2 libsigsegv@2.12 m4@1.4.18 mpfr@3.1.6 perl@5.30.2 readline@8.0 automake@1.16.2 gdbm@1.18.1 isl@0.20 libtool@2.4.6 mpc@1.1.0 ncurses@6.2 pkgconf@1.7.3 zlib@1.2.11 -- linux-centos7-skylake_avx512 / gcc@9.2.0 --------------------- autoconf@2.69 hwloc@1.11.11 libtool@2.4.6 netcdf-c@4.7.3 parallel-netcdf@1.12.1 util-macros@1.19.1 automake@1.16.2 libiconv@1.16 libxml2@2.9.10 netcdf-fortran@4.5.2 perl@5.30.2 xz@5.2.5 gdbm@1.18.1 libpciaccess@0.13.5 m4@1.4.18 numactl@2.0.12 pkgconf@1.7.3 zlib@1.2.11 hdf5@1.10.6 libsigsegv@2.12 ncurses@6.2 openmpi@3.1.6 readline@8.0 #. 安装 py-matplotlib@3.2.2 .. code-block:: bash $ spack install py-matplotlib@3.2.2 #. 安装 py-cartopy@0.17.0 .. code-block:: bash $ spack install py-cartopy@0.17.0 #. - 装载 openmpi@3.1.6 .. code-block:: bash $ spack load -r openmpi@3.1.6 .. note:: 将如下命令加入 :code:`~/.bashrc` 或 :code:`~/.cshrc` .. code-block:: bash spack load gcc@9.2.0 spack load -r openmpi@3.1.6 spack load -r py-matplotlib@3.2.2 spack load -r py-cartopy@0.17.0 安装 METplus ================================================ 详细安装细节请参照 `METplus文档 `_ 1. 以3.0.2版本为例: .. code-block:: bash $ cd METplus-3.0.2 # 下载稳定版本的MET（v9.0.2) 到 MET 目录 $ cd build_components #. 编辑 :code:`env_vars.bash` 如下 .. code-block:: bash export TEST_BASE=/home/wrf/METplus-3.0.2/build_components export COMPILER=gnu export MET_SUBDIR=${TEST_BASE} export MET_TARBALL=met.tar.gz export MET_PYTHON_CC=-I/usr/include/python3.6m export MET_PYTHON_LD="-L/usr/lib64 -lpython3.6m -lpthread -ldl -lutil -lm -Xlinker -export-dynamic" #### Python version on your system export PYTHON_MODULE=python_3.6.8 ### Local system ${PYTHON_MODULE} install location for python libraries export MET_PYTHON=/usr/local/python3.6 export PYTHON_MODULE_USE=1 export USE_MODULES=FALSE export SET_D64BIT=TRUE #. 编辑 :code:`build_MET.sh` 如下 .. code-block:: bash #!/bin/bash rm -fr met tar_files compile_MET_all.sh ### Grab the compenents source code from git using manage_externals ### Externals.cfg specifies what to checkout and where to put it #../manage_externals/checkout_externals ## Grab the compile script #wget https://dtcenter.org/sites/default/files/community-code/met/compile_scripts/compile_MET_all.sh.tgz tar -xzvf compile_MET_all.sh.tgz ## Grab the external library tar file #wget https://dtcenter.org/sites/default/files/community-code/met/compile_scripts/tar_files.tgz ## Stuff the contents into tar_files directory tar -xzvf tar_files.tgz ## link the git hub source code directory to the current directory - serious hack for now ln -s ../MET/met met ## Create a tarball which is what the compile script wants right now - hopefully change this later tar -cvzhf tar_files/met.tar.gz met ### Source environment variables and run the compile_all script source env_vars.bash cd met ./bootstrap cd ../ bash compile_MET_all.sh #. 手动下载 .. code-block:: bash wget https://dtcenter.org/sites/default/files/community-code/met/compile_scripts/compile_MET_all.sh.tgz wget https://dtcenter.org/sites/default/files/community-code/met/compile_scripts/tar_files.tgz # 存入 build_components 目录下 #. Hack :code:`compile_MET_all.sh` 删除 替换 :code:`_64BIT_` 的 :code:`sed` 语句，即：在编译 :code:`g2clib` 中使用 :code:`-D__64BIT__` 选项 #. 编译 .. code-block:: bash $ ./build_MET.sh $ ls -al bin total 208648 drwxrwxr-x. 2 wrf wrf 4096 Jun 21 12:00 . drwxrwxr-x. 7 wrf wrf 4096 Jun 21 12:35 .. -rwxr-xr-x. 1 wrf wrf 7410728 Jun 21 12:00 ascii2nc -rwxr-xr-x. 1 wrf wrf 8496616 Jun 21 12:00 ensemble_stat -rwxr-xr-x. 1 wrf wrf 6106912 Jun 21 12:00 gen_vx_mask -rwxr-xr-x. 1 wrf wrf 276400 Jun 21 12:00 gis_dump_dbf -rwxr-xr-x. 1 wrf wrf 357208 Jun 21 12:00 gis_dump_shp -rwxr-xr-x. 1 wrf wrf 293096 Jun 21 12:00 gis_dump_shx -rwxr-xr-x. 1 wrf wrf 6357248 Jun 21 12:00 grid_diag -rwxr-xr-x. 1 wrf wrf 8308912 Jun 21 12:00 grid_stat -rwxr-xr-x. 1 wrf wrf 4920136 Jun 21 12:00 gsid2mpr -rwxr-xr-x. 1 wrf wrf 4988592 Jun 21 12:00 gsidens2orank -rwxr-xr-x. 1 wrf wrf 1368040 Jun 21 12:00 lidar2nc -rwxr-xr-x. 1 wrf wrf 6909808 Jun 21 12:00 madis2nc -rwxr-xr-x. 1 wrf wrf 9338144 Jun 21 12:00 mode -rwxr-xr-x. 1 wrf wrf 3958560 Jun 21 12:00 mode_analysis -rwxr-xr-x. 1 wrf wrf 6923192 Jun 21 12:00 modis_regrid -rwxr-xr-x. 1 wrf wrf 7454832 Jun 21 12:00 mtd -rwxr-xr-x. 1 wrf wrf 7426312 Jun 21 12:00 pb2nc -rwxr-xr-x. 1 wrf wrf 5948184 Jun 21 12:00 pcp_combine -rwxr-xr-x. 1 wrf wrf 6080464 Jun 21 12:00 plot_data_plane -rwxr-xr-x. 1 wrf wrf 5770328 Jun 21 12:00 plot_point_obs -rwxr-xr-x. 1 wrf wrf 6257480 Jun 21 12:00 point2grid -rwxr-xr-x. 1 wrf wrf 8347488 Jun 21 12:00 point_stat -rwxr-xr-x. 1 wrf wrf 6112088 Jun 21 12:00 regrid_data_plane -rwxr-xr-x. 1 wrf wrf 6374800 Jun 21 12:00 rmw_analysis -rwxr-xr-x. 1 wrf wrf 7936008 Jun 21 12:00 series_analysis -rwxr-xr-x. 1 wrf wrf 5849264 Jun 21 12:00 shift_data_plane -rwxr-xr-x. 1 wrf wrf 9326944 Jun 21 12:00 stat_analysis -rwxr-xr-x. 1 wrf wrf 1794168 Jun 21 12:00 tc_dland -rwxr-xr-x. 1 wrf wrf 8351096 Jun 21 12:00 tc_gen -rwxr-xr-x. 1 wrf wrf 7153184 Jun 21 12:00 tc_pairs -rwxr-xr-x. 1 wrf wrf 6650656 Jun 21 12:00 tc_rmw -rwxr-xr-x. 1 wrf wrf 9035256 Jun 21 12:00 tc_stat -rwxr-xr-x. 1 wrf wrf 8573648 Jun 21 12:00 wavelet_stat -rwxr-xr-x. 1 wrf wrf 6598400 Jun 21 12:00 wwmca_plot -rwxr-xr-x. 1 wrf wrf 6519248 Jun 21 12:00 wwmca_regrid 安装 METviewer =================================== 1. 制作 `METviewer Docker 镜像 `_ 在一台安装了Docker和又互联网连接的机器上，下载 `METviewer Docker 镜像 `_ 制作 METviewer 的 Docker 镜像，存为 :code:`tar` 文件 .. code-block:: bash $ cd container-dtc-metviewer/METviewer $ docker image build -t metviewer . # 制作完成 $ docker image save metviewer -o metviewer.docker.image.tar #. 将该镜像文件上传到工作机器上 （同时也需要准备mysql的镜像文件），然后加载该镜像 .. code-block:: bash $ docker image load -i metviewer.docker.image.tar 0683de282177: Loading layer [==================================================>] 244.9MB/244.9MB 5714bb8688ee: Loading layer [==================================================>] 173MB/173MB ec559b28281c: Loading layer [==================================================>] 1.65GB/1.65GB 3335b9f0667b: Loading layer [==================================================>] 2.56kB/2.56kB 1fa73ec3c2f2: Loading layer [==================================================>] 27.03MB/27.03MB 8872ec41f650: Loading layer [==================================================>] 13.93MB/13.93MB 989c6a32b856: Loading layer [==================================================>] 44.75MB/44.75MB 16fa149a25ac: Loading layer [==================================================>] 2.048kB/2.048kB Loaded image: metviewer:latest $ docker image load -i mysql5.7.docker.image.tar 13cb14c2acd3: Loading layer [==================================================>] 72.49MB/72.49MB 365386a39e0e: Loading layer [==================================================>] 338.4kB/338.4kB c3f46b20a0d3: Loading layer [==================================================>] 9.539MB/9.539MB 66c45123fd43: Loading layer [==================================================>] 4.2MB/4.2MB 61cbb8ea6481: Loading layer [==================================================>] 1.536kB/1.536kB 44853bb67274: Loading layer [==================================================>] 53.75MB/53.75MB 3a2464d8e0c0: Loading layer [==================================================>] 6.656kB/6.656kB 91ae264962fb: Loading layer [==================================================>] 3.584kB/3.584kB 8f0182ef7c8c: Loading layer [==================================================>] 313.1MB/313.1MB ac7657905788: Loading layer [==================================================>] 16.38kB/16.38kB c90a34afcab0: Loading layer [==================================================>] 1.536kB/1.536kB Loaded image: mysql:5.7 $ docker image ls -a REPOSITORY TAG IMAGE ID CREATED SIZE metviewer latest 85b75697b9ee 9 days ago 2.11GB mysql 5.7 9cfcce23593a 11 days ago 448MB #. 在工作机器上，下载 `container-dtc-nwp `_ 和 `METviewer Docker 镜像 `_ 新建工作目录： .. code-block:: bash $ cd container-dtc-metviewer $ mkdir -p output $ mkdir -p mysql 编辑 :code:`docker-compose` 的配置文件 .. code-block:: bash $ cd container-dtc-nwp/components/metviewer/ $ vi docker-compose.yaml 修改路径（请注意高亮处）： .. code-block:: yaml :linenos: :emphasize-lines: 2,7,19-24 db: image: mysql:5.7 container_name: mysql_mv ports: - "6603:3306" volumes: - /home/wrf/container-dtc-metviewer/output/metviewer/mysql:/var/lib/mysql environment: MYSQL_ROOT_PASSWORD: mvuser tty: true # metviewer: image: metviewer container_name: metviewer ports: - "8080:8080" volumes: - /home/wrf/container-dtc-nwp/components/scripts:/scripts - /home/wrf/METout:/data - /home/wrf/container-dtc-metviewer/output/metviewer/output/xml:/opt/tomcat/webapps/metviewer_output/xml - /home/wrf/container-dtc-metviewer/output/metviewer/output/plots:/opt/tomcat/webapps/metviewer_output/plots - /home/wrf/container-dtc-metviewer/output/metviewer/output/data:/opt/tomcat/webapps/metviewer_output/data - /home/wrf/container-dtc-metviewer/output/metviewer/output/scripts:/opt/tomcat/webapps/metviewer_output/scripts links: - db:mysql_mv tty: true #. 启动 METviewer .. code-block:: bash $ cd container-dtc-nwp/components/metviewer/ $ docker-compose up -d Creating mysql_mv ... done Creating metviewer ... done 打开网页浏览器，输入网址 ：http://localhost:8080/metviewer/metviewer1.jsp 或 http://10.86.41.176:8080/metviewer/metviewer1.jsp .. figure:: images/metviewer_gui.PNG :scale: 30% :align: center .. note:: 在线图形显示可能有问题，请使用右上角的图形保持功能