##################################################### ## 如有转载，请务必保留本文链接及版权信息 ## 欢迎广大运维同仁一起交流 linux/unix网站运维技术! ##QQ:335623998 ##E-mail:335623998@qq.com ## 博客： h
##################################################### ##如有转载，请务必保留本文链接及版权信息 ##欢迎广大运维同仁一起交流linux/unix网站运维技术! ##QQ:335623998 ##E-mail:335623998@qq.com ##博客： http://dreamway.blog.51cto.com/ ##weibo：http://weibo.com/zhaixiangpan ##################################################### 运维工作中常会在linux命令行下载外网文件或内网进行大文件传输，经常使用的文本下载工具wget、curl，今天给大家推荐支持Linux命令行多线程、断点续传下载工具axel和myget。 1、系统环境 # lsb_release -a LSB Version: :core-4.0-amd64:core-4.0-ia32:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-ia32:printing-4.0-noarch Distributor ID: CentOS Description: CentOS release 5.8 (Final) Release: 5.8 Codename: Final 2、下载工具安装、使用方法介绍 2.1 wget CentOS 默认已经安装，如需安装请运行 # yum install wget -y wget版本信息 # wget -V GNU Wget 1.11.4 Red Hat modified 此工具比较常用，使用方法、参数略 2.2 Axel 下载 安装 官方站点http://axel.alioth.debian.org/ 源码包地址 http://alioth.debian.org/frs/?group_id=100070 最新版本axel-2.4源码包下载地址 http://alioth.debian.org/frs/download.php/3015/axel-2.4.tar.gz rpm包地址 http://pkgs.repoforge.org/axel/ 选择下载对应系统的rpm包 # wget http://pkgs.repoforge.org/axel/axel-2.4-1.el5.rf.x86_64.rpm 安装axel # rpm -ivh axel-2.4-1.el5.rf.x86_64.rpm axel版本 # axel -V Axel version 2.4 (Linux) axel命令使用方法： axel [选项参数] url1 [url2] [url……] axel 参数： --max-speed=x #限速值最高速度 -s x Specify maximum speed (bytes per second) --num-connections=x -n x #连接数 Specify maximum number of connections --output=f #下载为本地文件 -o f Specify local output file --search[=x] #搜索镜像 -S [x] Search for mirrors and download from x servers --header=x -H x #添加头文件字符串 Add header string --user-agent=x #设置UA -U x Set user agent --no-proxy #不使用代理服务器 -N Just don't use any proxy server --quiet --quiet, -q No output to stdout. #静默模式，不输出到标准输出 --verbose -v More status information #更多状态信息 --alternate --help #帮助 -h --version #版本 -V 2.3 myget 下载、安装 [root@cobbler-1014 ~]# wget http://myget.sourceforge.net/release/myget-0.1.2.tar.gz [root@cobbler-1014 ~]# tar zxf myget-0.1.2.tar.gz [root@cobbler-1014 ~]# cd myget-0.1.2 # ./configure && make && make install
mytget版本，注意myget命令为mytget
# mytget -v
myget 0.1.1
命令mytget用法 mytget [选项] [url] 参数 -b, --debug Show the debug message #看调试信息 -c, --count=num Set the retry count to [num], no limit when "0", the default is "99" #设置重试次数，0为无限，默认是99次。 -d, --directory=dir Set the local direcotry to [dir], the default is "." #指定下载本地目录，默认是当前目录 -f, --file=file Rename the file to [file] #重命名下载到本地的文件名 -h, --help A brief summary of all the options #简短的帮助摘要 -i, --interval=num Set the ftp retry interval to [num] seconds, the default is "5" #设置ftp重试间隔，单位s，默认5秒 -n, --number=num Use [num] connections instead of the default (4) #指定连接数，默认4 -r, --referer=URL Include `Referer: [URL]' header in HTTP request. #包含请求头 Referer -t, --timeout=num Set the connection timeout to [num] seconds, the default is "30" #设置连接超时时间，默认30秒 -v, --version Show the version of the myget and exit #查看版本信息 -x, --proxy=URL Set the proxy [URL] #设置代理 3、wget、axel、myget测试下载速度对比 3.1 下载速度对比 说明：同一台机器下载同一个数据源链接 3.1.1 wget下载 # wget http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso --2013-03-11 09:29:46-- http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Connecting to 10.0.251.154:80... connected. HTTP request sent, awaiting response... 200 OK Length: 4353378304 (4.1G) [application/octet-stream] Saving to: `CentOS-6.4-x86_64-bin-DVD1.iso' 0% [ ] 18,526,067 1.60M/s eta 46m 21s 3.2 axel下载 # axel http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Initializing download: http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso File size: 4353378304 bytes Opening output file CentOS-6.4-x86_64-bin-DVD1.iso.0 Starting download [ 0%] .......... .......... .......... .......... .......... [ 382.3KB/s] [ 0%] .......... .......... .......... .......... .......... [ 610.9KB/s] [ 0%] .......... .......... .......... .......... .......... [ 826.5KB/s] [ 0%] .......... .......... .......... .......... .......... [1003.1KB/s] 略 [ 0%] .......... .......... .......... .......... .......... [2055.8KB/s] [ 0%] .......... .......... .......... .......... .......... [2074.8KB/s] [ 0%] .......... .......... .......... .......... .......... [2093.6KB/s] [ 0%] .......... .......... .......... .......... .......... [2111.0KB/s] [ 0%] .......... .......... .......... .......... ... Downloaded 2693.8 kilobytes in 1 second. (2113.05 KB/s) # rm -f CentOS-6.4-x86_64-bin-DVD1.iso* 3.2.1 axel 使用10个线程下载 # axel -n 10 http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Initializing download: http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso File size: 4353378304 bytes Opening output file CentOS-6.4-x86_64-bin-DVD1.iso Starting download [ 0%] .......... .......... .......... .......... .......... [ 440.9KB/s] [ 0%] .......... .......... .......... .......... .......... [ 707.9KB/s] [ 0%] .......... .......... .......... .......... .......... [ 935.1KB/s] [ 0%] .......... .......... .......... .......... .......... [1138.8KB/s] [ 0%] .......... .......... .......... .......... .......... [1310.5KB/s] 略 [ 0%] .......... .......... .......... .......... .......... [2180.3KB/s] [ 0%] .......... .......... .......... .......... .......... [2196.9KB/s] [ 0%] .......... .......... .......... .......... .......... [2189.7KB/s] [ 0%] .......... .......... ......... Downloaded 4139.5 kilobytes in 1 second. (2176.65 KB/s) 3.3 myget下载 # mytget http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Begin to download: CentOS-6.4-x86_64-bin-DVD1.iso Filesize: 4.1G 0% [ > > > ] [ 27M] [2.5M/s] [ETA:26:58] 3.3.1 myget指定10个线程下载 # mytget -n 10 http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Begin to download: CentOS-6.4-x86_64-bin-DVD1.iso Filesize: 4.1G 0% [ > > > > > > > > > ] [ 39M] [2.6M/s] [ETA:26:27] # ll CentOS-6.4-x86_64-bin-DVD1.iso.mg\! -rw-r--r-- 1 root root 4353378404 Mar 11 12:53 CentOS-6.4-x86_64-bin-DVD1.iso.mg! 下载速度对比结论：axel、myget支持多线程，且速度较快都在2M。 4、断点续传对比测试 4.1 axel # axel -n 10 http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Initializing download: http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso File size: 4353378304 bytes Opening output file CentOS-6.4-x86_64-bin-DVD1.iso Starting download [ 0%] .......... .......... .......... .......... .......... [ 491.8KB/s] [ 0%] .......... .......... .......... .......... .......... [ 748.4KB/s] [ 0%] .......... .......... .......... .......... .......... [1007.5KB/s] 略 [ 1%] .......... ........... .......... .......... .......... [2509.3KB/s] [ 1%] .......... .......... .......... .......... .......... [2509.5KB/s] [ 1%] .......... .......... .......... .... Downloaded 45.3 megabytes in 18 seconds. (2510.95 KB/s) 中止下载，进度已经到1%，下载了45.3mb 再次运行下载命令，继续从1%断点续传 # axel -n 10 http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Initializing download: http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso File size: 4353378304 bytes Opening output file CentOS-6.4-x86_64-bin-DVD1.iso State file found: 47549233 bytes downloaded, 4305829071 to go. Starting download ,,,,,,,,,, ,,,,,,,,,, ,,,,,,,,,, ,,,,...... .......... [ 314.4KB/s] [ 1%] .......... .......... .......... .......... .......... [ 603.0KB/s] [ 1%] .......... .......... .......... .......... .......... [ 839.7KB/s] 略 [ 1%] .......... .......... .......... .......... .......... [1547.1KB/s] [ 1%] .......... .......... .......... .......... .......... [1548.8KB/s] # rm -f CentOS-6.4-x86_64-bin-DVD1.iso* 4.2 wget # wget http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso --2013-03-11 09:51:46-- http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Connecting to 10.0.251.154:80... connected. HTTP request sent, awaiting response... 200 OK Length: 4353378304 (4.1G) [application/octet-stream] Saving to: `CentOS-6.4-x86_64-bin-DVD1.iso' 1% [ ] 47,760,647 1.49M/s eta 47m 42s wget在进度1%中止已经下载47,760,647，再次运行命令，重新开始下载 # wget http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso --2013-03-11 09:53:24-- http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Connecting to 10.0.251.154:80... connected. HTTP request sent, awaiting response... 200 OK Length: 4353378304 (4.1G) [application/octet-stream] Saving to: `CentOS-6.4-x86_64-bin-DVD1.iso.1' 0% [ ] 1,032,445 1.21M/s wget直接运行不支持断点续传 # rm -f CentOS-6.4-x86_64-bin-DVD1.iso* wget重新测试使其支持断点续传 # wget http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso --2013-03-11 12:40:15-- http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Connecting to 10.0.251.154:80... connected. HTTP request sent, awaiting response... 200 OK Length: 4353378304 (4.1G) [application/octet-stream] Saving to: `CentOS-6.4-x86_64-bin-DVD1.iso' 1% [> ] 52,620,206 1.57M/s eta 44m 58s 在进度1%，已经下载52,620,206处停止 wget 加-c再次下载，支持断点续传 # wget -c http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso --2013-03-11 12:40:56-- http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Connecting to 10.0.251.154:80... connected. HTTP request sent, awaiting response... 206 Partial Content Length: 4353378304 (4.1G), 4300661972 (4.0G) remaining [application/octet-stream] Saving to: `CentOS-6.4-x86_64-bin-DVD1.iso' 1% [+ ] 59,663,080 1.59M/s eta 44m 14s # rm -f CentOS-6.4-x86_64-bin-DVD1.iso.mg\! 4.3 myget # mytget http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Begin to download: CentOS-6.4-x86_64-bin-DVD1.iso Filesize: 4.1G 1% [ > > > ] [ 45M] [2.1M/s] [ETA:33:06] 下载进度到1% 文件大小45M # ll CentOS-6.4-x86_64-bin-DVD1.iso.mg\! -rw-r--r-- 1 root root 4353378404 Mar 11 12:54 CentOS-6.4-x86_64-bin-DVD1.iso.mg! 再次执行命令，直接从进度1%开始下载 # mytget http://url/iso/Centos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso Begin to download: CentOS-6.4-x86_64-bin-DVD1.iso Filesize: 4.1G 1% [ > > > ] [ 51M] [2.9M/s] [ETA:23:57] 断点续传对比，三个工具都支持，但wget需要增加-c参数，axel、myget再次执行命令即可。 综上对比推荐大家工作中使用axel或myget提高效率，个人比较喜欢axel。 注意内网传输根据隧道带宽进行限速，别影响线上生产服务数据传输。 【文章原创作者：香港服务器 http://www.558idc.com/hk.html提供,感谢支持】