佚名 向大家分享了本人网站一次被黑的经历与反思 心得与体会,其中包含创业经历心得体会,澳门赌博经历心得,回顾入党经历心得体会,工作经历心得,重温入党经历心得体会等知识点,遇到此问题的同学们可以参考下
2012年5月22日17时许，发现公司一台非正式生产服务器（有公网IP）的root密码被修改，导致无法登陆服务器，排查处理过程如下：
1、通过vmware vcenter管理端进入单用户模式修改root密码
2、查看最近登陆信息，如下：
[root@localhost home]# last | more
root pts/5 218.247.13.60 Tue May 22 18:12 still logged in
root pts/4 120.72.48.70 Tue May 22 17:59 still logged in
root pts/3 120.72.48.70 Tue May 22 17:49 still logged in
root pts/2 218.247.13.60 Tue May 22 17:37 still logged in
root pts/1 218.247.13.60 Tue May 22 17:37 still logged in
root pts/0 218.247.13.60 Tue May 22 16:56 still logged in
root tty1 Tue May 22 16:56 - 16:56 (00:00)
reboot system boot 2.6.18-238.el5 Tue May 22 16:55 (01:56)
reboot system boot 2.6.18-238.el5 Tue May 22 16:53 (00:00)
lilei pts/3 120.72.48.52 Tue May 22 17:03 - down (00:05)
root pts/0 188.173.171.146 Tue May 22 16:16 - 16:30 (00:13)
发现16:16分来自罗马尼亚的IP（IP138资源）有登陆
立刻查看secure日志，如下
May 22 16:16:46 localhost sshd[26364]: Address 188.173.171.146 maps to 188-173-171-146.next-
gen.ro, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
May 22 16:16:48 localhost sshd[26364]: Accepted password for root from 188.173.171.146 port
1493 ssh2
May 22 16:16:48 localhost sshd[26364]: pam_unix(sshd:session): session opened for user root by
(uid=0)
May 22 16:17:09 localhost passwd: pam_unix(passwd:chauthtok): password changed for root
May 22 16:24:53 localhost sshd[3521]: Received signal 15; terminating.
May 22 16:34:51 localhost sshd[26364]: pam_unix(sshd:session): session closed for user root
May 22 16:39:47 localhost groupadd[1622]: new group: name=screen, GID=84
可以确定此机已经被黑，首先将此IP 188.173.171.146加入hosts.deny防止在处理过程中再次破坏，通过日志可以看出，被建立了一个组screen gid为84，在/etc/group中找到删除，继续排查。
由于此机器非正式环境使用，安全方面无过多策略，只开启http下载服务，扫描端口如下：
[root@localhost home]# nmap 127.0.0.1
Starting Nmap 4.11 ( http://www.insecure.org/nmap/ ) at 2012-05-23 15:13 CST
Interesting ports on localhost.localdomain (127.0.0.1):
Not shown: 1676 closed ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
80/tcp open http
111/tcp open rpcbind
900/tcp open unknown
ps查看进程如下
[root@localhost home]# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 3676 0.0 0.0 74836 1236 ? S 16:56 0:00 crond
xfs 3699 0.0 0.0 20108 1044 ? S 16:56 0:00 xfs -droppriv -da
avahi 3750 0.0 0.0 23172 1284 ? S 16:56 0:00 avahi-daemon: run
avahi 3751 0.0 0.0 23172 340 ? S 16:56 0:00 avahi-daemon: chr
root 3883 0.0 0.0 18440 480 ? S 16:56 0:00 /usr/sbin/smartd
root 3888 0.0 0.0 3816 492 tty2 S 16:56 0:00 /sbin/mingetty tt
root 3890 0.0 0.0 3816 492 tty3 S 16:56 0:00 /sbin/mingetty tt
root 3891 0.0 0.0 3816 484 tty4 S 16:56 0:00 /sbin/mingetty tt
root 3892 0.0 0.0 3816 488 tty5 S 16:56 0:00 /sbin/mingetty tt
root 3893 0.0 0.0 3816 484 tty6 S 16:56 0:00 /sbin/mingetty tt
root 3959 0.0 0.8 258352 16992 ? SN 16:56 0:00 /usr/bin/python -
root 3961 0.0 0.0 12940 1192 ? SN 16:56 0:00 /usr/libexec/gam_
root 4024 0.0 0.0 3816 492 tty1 S 16:56 0:00 /sbin/mingetty tt
root 4025 0.0 0.0 24068 1740 ? S 16:56 0:00 sshd: root@pts/0,
root 4036 0.0 0.0 66088 1580 pts/0 S 16:56 0:00 -bash
root 4944 0.5 0.2 157528 5196 ? S 17:24 0:28 /usr/bin/python /
root 5170 0.0 0.0 66084 1484 pts/1 S 17:37 0:00 -bash
root 5200 0.0 0.0 24068 1696 ? S 17:37 0:00 sshd: root@pts/2
root 5208 0.0 0.0 66088 1536 pts/2 S 17:37 0:00 -bash
root 5341 0.0 0.0 23904 1688 ? S 17:49 0:00 sshd: root@pts/3
root 5349 0.0 0.0 66088 1572 pts/3 S 17:49 0:00 -bash
root 5457 0.0 0.0 23904 1548 ? S 17:59 0:00 sshd: root@pts/4
root 5465 0.0 0.0 66084 1484 pts/4 S 17:59 0:00 -bash
root 5591 0.0 0.0 24068 1704 ? S 18:12 0:00 sshd: root@pts/5
root 5599 0.0 0.0 66088 1568 pts/5 S 18:12 0:00 -bash
root 5895 0.0 0.1 4200 2092 pts/5 R 18:53 0:00 ps aux
从进程并未看出有什么异常进程，继续排查
查看日志目录，发现www.jb51.net maillog日志异常大，内容如下：
)
May 22 16:53:48 localhost sendmail[3647]: q4M8rjFf003627: to=<bloodvio@yahoo.com>, ctladdr=<root@localhost.localdomain> (0/0), delay=00:00:03, xdelay=00:00:03, mailer=esmtp, pri=120382, relay=mta5.am0.yahoodns.net. [66.94.238.147], dsn=2.0.0, stat=Sent (ok dirdel)
May 22 16:56:30 localhost sendmail[3965]: q4M8uU5j003965: to=bloodvio@yahoo.com, ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=30083, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (q4M8uUoA003985 Message accepted for delivery)
May 22 16:56:32 localhost sendmail[3987]: q4M8uUoA003985: to=<bloodvio@yahoo.com>, ctladdr=<root@localhost.localdomain> (0/0), delay=00:00:02, xdelay=00:00:02, mailer=esmtp, pri=120381, relay=mta5.am0.yahoodns.net. [67.195.103.233], dsn=2.0.0, stat=Sent (ok dirdel)
May 22 16:56:43 localhost sendmail[4030]: q4M8uhGd004030: to=bloodvio@yahoo.com, ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=30083, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (q4M8uhR3004033 Message accepted for delivery)
May 22 16:56:46 localhost sendmail[4035]: q4M8uhR3004033: to=<bloodvio@yahoo.com>, ctladdr=<root@localhost.localdomain> (0/0), delay=00:00:03, xdelay=00:00:03, mailer=esmtp, pri=120381, relay=mta5.am0.yahoodns.net. [72.30.235.196], dsn=2.0.0, stat=Sent (ok dirdel)
可以看出，在频繁给yahoo发邮件，本以为此人只为盗发邮件才入侵我的机器，但是仔细一看，以前被盗发邮件都发送给不同账号，但这个是同一账号，应该是黑客接收邮件的客户端。
继续排查。
查看定时任务：
[root@localhost mabil]# ls /var/spool/crron
root
[root@localhost mail]# crontab -l
0 6 * * * /usr/sbin/ntpdate asia.pool.ntp.org >>/var/log/ntpdatelog
没有异常的定时任务
查看用户
[root@localhost mail]# cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/bin/bash
oprofile:x:16:16:Special user account to be used by OProfile:/home/oprofile:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
xfs:x:43:43:X Font Server:/etc/X11/fs:/sbin/nologin
rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin
nfsnobody:x:4294967294:4294967294:Anonymous NFS User:/var/lib/nfs:/sbin/nologin
haldaemon:x:68:68:HAL daemon:/:/sbin/nologin
avahi-autoipd:x:100:156:avahi-autoipd:/var/lib/avahi-autoipd:/sbin/nologin
apache:x:48:48:Apache:/var/www:/sbin/nologin
nagios:x:502:501::/home/nagios:/bin/bash
异常：仔细看一下bin用户的登陆shell，默认应该为/sbin/nologin，它为/bin/bash，也就是意味着它可以通过密码登陆系统，改之。
查看group
root:x:0:root
bin:x:1:root,bin,daemon
daemon:x:2:root,bin,daemon
sys:x:3:root,bin,adm
adm:x:4:root,adm,daemon
tty:x:5:
nagios:x:501:
screen:x:84:
除了最后一行screen（已经在日志中提示）异常，其他无异常，删之
到了此时，不知道如何排查了，感觉被黑以后并没有对服务器做过多操作，top盯着看看吧，有什么特殊进程：啊哈，看到了，不是进程，是显示，
6:53pm up 1:59, 6 users, load average: 0.00, 0.00, 0.00
102 processes: 101 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: 0.7% user, 0.6% system, 0.0% nice, 98.5% idle
Mem: 2058840K av, 613508K used, 1445332K free, 0K shrd, 86528K buff
Swap: 2064376K av, 0K used, 2064376K free 327072K cached
$<5>$<3>$<2>$<2>Unknown command `
' -- hit `h' for help$<2>IZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND
1 root 15 0 10372 688 576 S 0 0.0 0.0 0:00 init
2 root 0K -5 0 0 0 SW< 0
1、通过vmware vcenter管理端进入单用户模式修改root密码
2、查看最近登陆信息，如下：
[root@localhost home]# last | more
root pts/5 218.247.13.60 Tue May 22 18:12 still logged in
root pts/4 120.72.48.70 Tue May 22 17:59 still logged in
root pts/3 120.72.48.70 Tue May 22 17:49 still logged in
root pts/2 218.247.13.60 Tue May 22 17:37 still logged in
root pts/1 218.247.13.60 Tue May 22 17:37 still logged in
root pts/0 218.247.13.60 Tue May 22 16:56 still logged in
root tty1 Tue May 22 16:56 - 16:56 (00:00)
reboot system boot 2.6.18-238.el5 Tue May 22 16:55 (01:56)
reboot system boot 2.6.18-238.el5 Tue May 22 16:53 (00:00)
lilei pts/3 120.72.48.52 Tue May 22 17:03 - down (00:05)
root pts/0 188.173.171.146 Tue May 22 16:16 - 16:30 (00:13)
发现16:16分来自罗马尼亚的IP（IP138资源）有登陆
立刻查看secure日志，如下
May 22 16:16:46 localhost sshd[26364]: Address 188.173.171.146 maps to 188-173-171-146.next-
gen.ro, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
May 22 16:16:48 localhost sshd[26364]: Accepted password for root from 188.173.171.146 port
1493 ssh2
May 22 16:16:48 localhost sshd[26364]: pam_unix(sshd:session): session opened for user root by
(uid=0)
May 22 16:17:09 localhost passwd: pam_unix(passwd:chauthtok): password changed for root
May 22 16:24:53 localhost sshd[3521]: Received signal 15; terminating.
May 22 16:34:51 localhost sshd[26364]: pam_unix(sshd:session): session closed for user root
May 22 16:39:47 localhost groupadd[1622]: new group: name=screen, GID=84
可以确定此机已经被黑，首先将此IP 188.173.171.146加入hosts.deny防止在处理过程中再次破坏，通过日志可以看出，被建立了一个组screen gid为84，在/etc/group中找到删除，继续排查。
由于此机器非正式环境使用，安全方面无过多策略，只开启http下载服务，扫描端口如下：
[root@localhost home]# nmap 127.0.0.1
Starting Nmap 4.11 ( http://www.insecure.org/nmap/ ) at 2012-05-23 15:13 CST
Interesting ports on localhost.localdomain (127.0.0.1):
Not shown: 1676 closed ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
80/tcp open http
111/tcp open rpcbind
900/tcp open unknown
ps查看进程如下
[root@localhost home]# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 3676 0.0 0.0 74836 1236 ? S 16:56 0:00 crond
xfs 3699 0.0 0.0 20108 1044 ? S 16:56 0:00 xfs -droppriv -da
avahi 3750 0.0 0.0 23172 1284 ? S 16:56 0:00 avahi-daemon: run
avahi 3751 0.0 0.0 23172 340 ? S 16:56 0:00 avahi-daemon: chr
root 3883 0.0 0.0 18440 480 ? S 16:56 0:00 /usr/sbin/smartd
root 3888 0.0 0.0 3816 492 tty2 S 16:56 0:00 /sbin/mingetty tt
root 3890 0.0 0.0 3816 492 tty3 S 16:56 0:00 /sbin/mingetty tt
root 3891 0.0 0.0 3816 484 tty4 S 16:56 0:00 /sbin/mingetty tt
root 3892 0.0 0.0 3816 488 tty5 S 16:56 0:00 /sbin/mingetty tt
root 3893 0.0 0.0 3816 484 tty6 S 16:56 0:00 /sbin/mingetty tt
root 3959 0.0 0.8 258352 16992 ? SN 16:56 0:00 /usr/bin/python -
root 3961 0.0 0.0 12940 1192 ? SN 16:56 0:00 /usr/libexec/gam_
root 4024 0.0 0.0 3816 492 tty1 S 16:56 0:00 /sbin/mingetty tt
root 4025 0.0 0.0 24068 1740 ? S 16:56 0:00 sshd: root@pts/0,
root 4036 0.0 0.0 66088 1580 pts/0 S 16:56 0:00 -bash
root 4944 0.5 0.2 157528 5196 ? S 17:24 0:28 /usr/bin/python /
root 5170 0.0 0.0 66084 1484 pts/1 S 17:37 0:00 -bash
root 5200 0.0 0.0 24068 1696 ? S 17:37 0:00 sshd: root@pts/2
root 5208 0.0 0.0 66088 1536 pts/2 S 17:37 0:00 -bash
root 5341 0.0 0.0 23904 1688 ? S 17:49 0:00 sshd: root@pts/3
root 5349 0.0 0.0 66088 1572 pts/3 S 17:49 0:00 -bash
root 5457 0.0 0.0 23904 1548 ? S 17:59 0:00 sshd: root@pts/4
root 5465 0.0 0.0 66084 1484 pts/4 S 17:59 0:00 -bash
root 5591 0.0 0.0 24068 1704 ? S 18:12 0:00 sshd: root@pts/5
root 5599 0.0 0.0 66088 1568 pts/5 S 18:12 0:00 -bash
root 5895 0.0 0.1 4200 2092 pts/5 R 18:53 0:00 ps aux
从进程并未看出有什么异常进程，继续排查
查看日志目录，发现www.jb51.net maillog日志异常大，内容如下：
)
May 22 16:53:48 localhost sendmail[3647]: q4M8rjFf003627: to=<bloodvio@yahoo.com>, ctladdr=<root@localhost.localdomain> (0/0), delay=00:00:03, xdelay=00:00:03, mailer=esmtp, pri=120382, relay=mta5.am0.yahoodns.net. [66.94.238.147], dsn=2.0.0, stat=Sent (ok dirdel)
May 22 16:56:30 localhost sendmail[3965]: q4M8uU5j003965: to=bloodvio@yahoo.com, ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=30083, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (q4M8uUoA003985 Message accepted for delivery)
May 22 16:56:32 localhost sendmail[3987]: q4M8uUoA003985: to=<bloodvio@yahoo.com>, ctladdr=<root@localhost.localdomain> (0/0), delay=00:00:02, xdelay=00:00:02, mailer=esmtp, pri=120381, relay=mta5.am0.yahoodns.net. [67.195.103.233], dsn=2.0.0, stat=Sent (ok dirdel)
May 22 16:56:43 localhost sendmail[4030]: q4M8uhGd004030: to=bloodvio@yahoo.com, ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=30083, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (q4M8uhR3004033 Message accepted for delivery)
May 22 16:56:46 localhost sendmail[4035]: q4M8uhR3004033: to=<bloodvio@yahoo.com>, ctladdr=<root@localhost.localdomain> (0/0), delay=00:00:03, xdelay=00:00:03, mailer=esmtp, pri=120381, relay=mta5.am0.yahoodns.net. [72.30.235.196], dsn=2.0.0, stat=Sent (ok dirdel)
可以看出，在频繁给yahoo发邮件，本以为此人只为盗发邮件才入侵我的机器，但是仔细一看，以前被盗发邮件都发送给不同账号，但这个是同一账号，应该是黑客接收邮件的客户端。
继续排查。
查看定时任务：
[root@localhost mabil]# ls /var/spool/crron
root
[root@localhost mail]# crontab -l
0 6 * * * /usr/sbin/ntpdate asia.pool.ntp.org >>/var/log/ntpdatelog
没有异常的定时任务
查看用户
[root@localhost mail]# cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/bin/bash
oprofile:x:16:16:Special user account to be used by OProfile:/home/oprofile:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
xfs:x:43:43:X Font Server:/etc/X11/fs:/sbin/nologin
rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin
nfsnobody:x:4294967294:4294967294:Anonymous NFS User:/var/lib/nfs:/sbin/nologin
haldaemon:x:68:68:HAL daemon:/:/sbin/nologin
avahi-autoipd:x:100:156:avahi-autoipd:/var/lib/avahi-autoipd:/sbin/nologin
apache:x:48:48:Apache:/var/www:/sbin/nologin
nagios:x:502:501::/home/nagios:/bin/bash
异常：仔细看一下bin用户的登陆shell，默认应该为/sbin/nologin，它为/bin/bash，也就是意味着它可以通过密码登陆系统，改之。
查看group
root:x:0:root
bin:x:1:root,bin,daemon
daemon:x:2:root,bin,daemon
sys:x:3:root,bin,adm
adm:x:4:root,adm,daemon
tty:x:5:
nagios:x:501:
screen:x:84:
除了最后一行screen（已经在日志中提示）异常，其他无异常，删之
到了此时，不知道如何排查了，感觉被黑以后并没有对服务器做过多操作，top盯着看看吧，有什么特殊进程：啊哈，看到了，不是进程，是显示，
6:53pm up 1:59, 6 users, load average: 0.00, 0.00, 0.00
102 processes: 101 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: 0.7% user, 0.6% system, 0.0% nice, 98.5% idle
Mem: 2058840K av, 613508K used, 1445332K free, 0K shrd, 86528K buff
Swap: 2064376K av, 0K used, 2064376K free 327072K cached
$<5>$<3>$<2>$<2>Unknown command `
' -- hit `h' for help$<2>IZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND
1 root 15 0 10372 688 576 S 0 0.0 0.0 0:00 init
2 root 0K -5 0 0 0 SW< 0