存档十月 2019

开启SR-IOV遇到bus 02 out of range of [bus 01]

硬件配置,在此。

主板要求开启vt-x,vt-d,cpu必须有vt-d选项,通常i3不支持。查看CPU支持特征查询

SR-IOV bus 02 out of range of [bus 01]

报错:SR-IOV: bus number out of range,那是因为主板比较老不支持SR-IOV去设置给每个pci卡分配中断号,可使用pci=assign-busses参数强制分配。

#echo "options igb max_vfs=3" > /etc/modprobe.d/igb.conf
#update-initramfs -u
root@i:~# cat /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs
3
root@i:~# 
root@i:~# cat /etc/default/grub
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=downstream pci=assign-busses"
GRUB_CMDLINE_LINUX=""

修改igb.conf的值后,需要执行更新命令 update-initramfs -u,否则不生效。

看到虚拟功能的82576
root@i:~# dmesg | grep igb
[    1.653018] igb 0000:01:00.0: enabling device (0000 -> 0002)
[    1.654267] igb: 0000:01:00.0: igb_validate_option: max_vfs - SR-IOV VF devices set to 3
[    1.761083] igb 0000:01:00.0: SR-IOV has been enabled: configure port VLANs to keep your VFs secure
[    1.762045] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
[    1.762047] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    1.985184] igb 0000:01:00.0: added PHC on eth0
[    1.985189] igb 0000:01:00.0: Intel(R) Gigabit Ethernet Linux Driver
[    1.985193] igb 0000:01:00.0: eth0: (PCIe:2.5GT/s:Width x4) 
[    1.985198] igb 0000:01:00.0 eth0: MAC: 00:1b:22:56:b3:04
[    1.985277] igb 0000:01:00.0: eth0: PBA No: E43709-003
[    1.985281] igb 0000:01:00.0: LRO is disabled
[    1.985285] igb 0000:01:00.0: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s)
[    1.985353] igb 0000:01:00.1: enabling device (0000 -> 0002)
[    2.205117] igb 0000:01:00.1: added PHC on eth1
[    2.205122] igb 0000:01:00.1: Intel(R) Gigabit Ethernet Linux Driver
[    2.205126] igb 0000:01:00.1: eth1: (PCIe:2.5GT/s:Width x4) 
[    2.205131] igb 0000:01:00.1 eth1: MAC: 00:1b:22:56:b3:05
[    2.205211] igb 0000:01:00.1: eth1: PBA No: E43709-003
[    2.205215] igb 0000:01:00.1: LRO is disabled
[    2.205220] igb 0000:01:00.1: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s)
[    2.205303] igbvf 0000:02:10.0: enabling device (0000 -> 0002)
[    2.206637] igbvf 0000:02:10.0: PF still in reset state. Is the PF interface up?
[    2.206640] igbvf 0000:02:10.0: Assigning random MAC address.
[    2.206646] igb 0000:01:00.1 enp1s0f1: renamed from eth1
[    2.207823] igbvf 0000:02:10.0: PF still resetting
[    2.244508] igbvf 0000:02:10.0: Intel(R) 82576 Virtual Function
[    2.244511] igbvf 0000:02:10.0: Address: 82:a1:cf:82:43:35
[    2.244568] igbvf 0000:02:10.2: enabling device (0000 -> 0002)
[    2.244740] igb 0000:01:00.0 enp1s0f0: renamed from eth0
[    2.246089] igbvf 0000:02:10.2: PF still in reset state. Is the PF interface up?
[    2.246092] igbvf 0000:02:10.2: Assigning random MAC address.

iommu不能分组

If you don’t have dedicated iommu, your can try :


1) move the card to another pci slot

2) #nano /etc/default/grub

3) add “pcie_acs_override=downstream” to grub options, which can help on some setup with bad ACS implementation.

增加pcie_acs_override=downstream,而后只需更新启动。

#update-grub

分组后,虚拟化的网卡,可以分配给不同的VM。 勾选全部功能,则不能分别指派网卡。

Why does SR-IOV feature work only with the specific PCIE slots for ASUS P10S series platform?

华硕主板P10S系列,只有特定的PCIE槽支持SR-IOV特征。

Why does SR-IOV feature work only with the specific PCIE slots for P10S series platform?

According to hardware design, P10S series platform supports SR-IOV feature only by the PCIE slots which are controlled by PCH. In other words, the rest of PCIE slots, connected directly to CPU, don’t have SR-IOV function.

The following chat introduces each model of P10S series supports SR-IOV function by the corresponding PCIE slots. Please follow up the usage when you need to use virtual functions.

ModelPCIE slot
(SR-IOV supported)
Link
P10S-EPCIE4x4
P10S-CN/AN/A
P10S-VPCIE5x4
P10S-XPCIE5x4
P10S-MPCIE4x4
P10S-M-DCPCIE7x4
P10S-IN/AN/A
P10S-M WS/IPMI-OPCIE4x4
P10S-M WSPCIE4x4
P10S WSPCIE3/4x4

如文中强调,直连CPU的PCIE不可以用,如插显卡的PCIE16X。

proxmox 5.4安装hostapd服务

安装最新版hostapd,不使用apt install hostapd命令进行安装,因为apt方式安装的不是最新版。

proxmox 5.4,主板自带一有线网卡,后加一无线网卡AR9382,半高mini pice。

抓取到服务已启动
wlp3s0状态是活动的
root@i:~# systemctl status hostapd.service
● hostapd.service - hostapd service
   Loaded: loaded (/etc/systemd/system/hostapd.service; enabled; vendor preset: enabled)
   Active: active (exited) since Sat 2019-10-26 10:26:08 CST; 9s ago
  Process: 2950 ExecStart=/bin/bash -c /usr/local/bin/hostapd -B /etc/hostapd.conf  (code=exited, status=0/SUCCESS)
 Main PID: 2950 (code=exited, status=0/SUCCESS)
    Tasks: 1 (limit: 4915)
   Memory: 752.0K
      CPU: 53ms
   CGroup: /system.slice/hostapd.service
           └─2964 /usr/local/bin/hostapd -B /etc/hostapd.conf

Oct 26 10:26:08 i systemd[1]: Starting hostapd service...
Oct 26 10:26:08 i bash[2950]: Configuration file: /etc/hostapd.conf
Oct 26 10:26:08 i bash[2950]: wlp3s0: interface state UNINITIALIZED->COUNTRY_UPDATE
Oct 26 10:26:08 i bash[2950]: Using interface wlp3s0 with hwaddr e0:ca:94:28:93:58 and ssid "mywifi"
Oct 26 10:26:08 i bash[2950]: wlp3s0: interface state COUNTRY_UPDATE->ENABLED
Oct 26 10:26:08 i bash[2950]: wlp3s0: AP-ENABLED
Oct 26 10:26:08 i systemd[1]: Started hostapd service.

proxmox默认以root用户登录,使用systemd添加hostapd的自启动服务,简单、简单。。。

确定hostapd的绝对路径

root@i:~# whereis hostapd
hostapd: /etc/hostapd.conf /usr/local/bin/hostapd

创建文件:/etc/systemd/system/hostapd.service

root@i:~# cat /etc/systemd/system/hostapd.service
[Unit]
Description=hostapd service

[Service]
ExecStart=/bin/bash -c "/usr/local/bin/hostapd -B /etc/hostapd.conf "
Type=oneshot
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

注:/etc/hostapd.conf配置文件,是自己新建的,hostapd没有带example。

nano编辑,保存后退出,运行命令:

sudo systemctl daemon-reload
sudo systemctl enable hostapd.service
root@i:~# systemctl list-unit-files | grep hostapd
hostapd.service                        enabled

重启系统,hostapd自动启动。

可以使用systemctl停止、启动及重启hostapd服务。

systemctl start hostapd.service
systemctl stop hostapd.service
systemctl restart hostapd.service

赏心悦目。。。

proxmox 5.4安装hostapd+dhcp

网络配置:1个板载网卡intel I217-V,1个ar9382无线网卡。

I217-V计划连接光猫,ar9382则搭建AP,提供PVE的设置服务。

root@i:~# cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

auto vmbr0
iface vmbr0 inet static
        address  192.168.1.10
        netmask  255.255.255.0
        gateway  192.168.1.1
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
#lan

iface wlp3s0 inet manual

auto vmbr1
iface vmbr1 inet static
        address  10.0.0.1
        netmask  255.255.255.0
        bridge-ports wlp3s0
        bridge-stp off
        bridge-fd 0

正确的/etc/network/interfaces内容。

root@i:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
    link/ether 08:62:66:a4:87:9f brd ff:ff:ff:ff:ff:ff
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr1 state UP group default qlen 1000
    link/ether e0:ca:94:28:93:58 brd ff:ff:ff:ff:ff:ff
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 08:62:66:a4:87:9f brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.10/24 brd 192.168.1.255 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::a62:66ff:fea4:879f/64 scope link 
       valid_lft forever preferred_lft forever
5: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e0:ca:94:28:93:58 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/24 brd 10.0.0.255 scope global vmbr1
       valid_lft forever preferred_lft forever
    inet6 fe80::8c8a:e8ff:fed9:808e/64 scope link 
       valid_lft forever preferred_lft forever

ip地址内容。

root@i:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 08:62:66:a4:87:9f brd ff:ff:ff:ff:ff:ff
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr1 state UP mode DEFAULT group default qlen 1000
    link/ether e0:ca:94:28:93:58 brd ff:ff:ff:ff:ff:ff
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 08:62:66:a4:87:9f brd ff:ff:ff:ff:ff:ff
5: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether e0:ca:94:28:93:58 brd ff:ff:ff:ff:ff:ff

正确的ip link状态

创建2个桥接
root@i:~# cat /ect/hostapd
cat: /ect/hostapd: No such file or directory
root@i:~# cat /ect/hostapd.conf
cat: /ect/hostapd.conf: No such file or directory
root@i:~# cat /etc/hostapd.conf
interface=wlp3s0
bridge=vmbr1
driver=nl80211
#driver=mac80211
country_code=US
ssid=mywifi
hw_mode=g
channel=11

wpa=2
# WPA2 only
wpa_passphrase=12345678
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP
auth_algs=1
macaddr_acl=0

# 802.11n
wmm_enabled=1
#ieee80211n=1
#ht_capab=[HT40-][SHORT-GI-20][SHORT-GI-40][DSSS_CCK-40][DSSS_CCK-40][DSSS_CCK-40]

注意第6行指定了桥接的界面,否则客户端不能拿到dhcp地址。安装hostapd参照。

安装dhcp服务。

sudo apt install isc-dhcp-server

指定dhcp监听界面:

root@i:~# cat /etc/default/isc-dhcp-server
# Defaults for isc-dhcp-server (sourced by /etc/init.d/isc-dhcp-server)

# Path to dhcpd's config file (default: /etc/dhcp/dhcpd.conf).
#DHCPDv4_CONF=/etc/dhcp/dhcpd.conf
#DHCPDv6_CONF=/etc/dhcp/dhcpd6.conf

# Path to dhcpd's PID file (default: /var/run/dhcpd.pid).
#DHCPDv4_PID=/var/run/dhcpd.pid
#DHCPDv6_PID=/var/run/dhcpd6.pid

# Additional options to start dhcpd with.
#       Don't use options -cf or -pf here; use DHCPD_CONF/ DHCPD_PID instead
#OPTIONS=""

# On what interfaces should the DHCP server (dhcpd) serve DHCP requests?
#       Separate multiple interfaces with spaces, e.g. "eth0 eth1".
INTERFACESv4="vmbr1"
#INTERFACESv6="vmbr1"

简单dhcp配置,/etc/dhcp/dhcpd.conf

subnet 10.0.0.0 netmask 255.255.255.0 {
 range 10.0.0.100 10.0.0.200;
 option domain-name-servers 1.1.1.1;
 option domain-name "thingy.top";
}

启动hostapd服务。

hostapd -d -B /etc/hostapd.conf

连接AP后,可以访问 https://10.0.0.1:8006来管理PVE。

添加hostapd为开机自动启动。

// 在/etc/init.d位置,新建hostapd.sh文件
root@i:/etc/init.d# cat hostapd.sh
/usr/local/bin/hostapd -d -B /etc/hostapd.conf

root@i:/etc/init.d# 
chmod 777 start.sh  //修改权限
// 增加开机脚本
update-rc.d startsh defaults 100

ubuntu编译安装hostapd最新版

下载

官方网站

wget https://w1.fi/releases/hostapd-2.9.tar.gz

解压

tar zxvf hostapd-2.9.tar.gz

安装依赖

sudo apt install libnl-3-dev
sudo apt install libnl-genl-3-dev
sudo apt install libssl-dev  // openssl
sudo apt install build-essential  // make编译

编译

cd hostapd-2.9/hostapd
cp defconfig .config
make
sudo make install

使用hostapd -v查看版本。

推荐: BT天堂

使用ProXmoX VE的虚拟化家庭服务器

1、CPU

选择e3 1220L V3,TDP 13W,双核4线程,1.1G HZ,最高睿频至1.3GHZ。 在PE环境下,最高频率有到1.5GHZ。 买个便宜的就好,130元。

优点是它不仅支持VT-x,还支持VT-d。

缺点是,无显卡,需要一块亮机卡。

2、主板

期望是一个DC供电主板,sata数量至少2个,有pci-e 1x插槽。

华硕H81M-K/K31ADE
华硕H81M-K/K31ADE

如果,1220L不能满足将来需求,可以方便的升级到更高的CPU。这也是选择1150针主板,而不是1155主板的原因。

3、内存

4G DDR3 1600内存,不贵,70元1条。

他家内存还好,条子有fru标签

可以买2条,这里只使用1条。ESXi需要至少4G否则不启动,proxmox没要求。内存大点好。买山寨条,不如买拆机,旧点不影响使用。

4、电源

买19V 120W的,余量足一些。华硕H81M-K/K31ADE,可以使用飞行堡垒的笔记本电源,插头是5.5*2.5的。台达的ADP-90MDH,插头尺寸对,功率是90W。价格要169元,嫌贵。

华硕飞行堡垒A550J FX-PRO FX-PLUS FX503V FX53V N56V N53S N55S电源适配器19V6.32A 120W充电器 PA1121-28

或者买建兴的,135W。

负载在60~80%时,电源效率高。只要电源电压是19V、19.5V,插头是5.5*2.5的,都可以拿来用。不使用显卡,或者使用GT720这种19W的低功耗显卡,也可以使用65W电源。

5、机箱

乔思伯机箱,背面的电源,没法封堵,比较难看,而且铝合金边缘割手,一丝血痕算是滴血认主。

华硕H81M-K/K31ADE板子尺寸22.6*17.5cm,乔思伯C2完全装的下。

乔思伯C2
注意 电源位置我们用不上,这里得填封一下。

怕安不进 华硕H81M-K/K31ADE ,结果是买了一个乔思伯V4.

只能说V4,太大了。

总花费:

747元。

键盘/鼠标/显示器,从别的地方借来用用。

不插显卡,没有核显,主板能正常启动系统吗?

做家用虚拟化服务器,显卡在不需要的情况下,拔掉可以降低系统的功耗,节省出来的pcie 16x插槽可以扩展1个双口网卡:数字鱼PCIE千兆网卡 双口82576E1G42ET服务器网卡。

那么,没有显卡,CPU也无核显,能够启动进系统吗?

答案是:

能。

需要修改BIOS,避免出现错误后停机。

华硕主板,如H81,在其BIOS Boot启动选项,禁用 wait For ‘ F1’ If Error

修改 wait For ‘ F1’ If Error ( wait for f1 if error ), Enabled —> Disabled, F10保存。

其他陈旧的BIOS,在main里修改, Halt on no errors

先插上显卡,安装系统。测试、配置正常后,拔掉显卡。 通过ssh、或者webGUI访问服务器配置。

华硕H81M-K/K31ADE H81主板

全新一年保,Asus/华硕H81M-K/K31ADE H81主板,1150针,DC供电HDMI,DDR3

Asus/华硕H81M-K/K31ADE H81主板1150针DC供电主板(DC供电的主板少之又少,数量不多机会把握)HDMI/DDR3,2个内存插槽,支持VGA+HDMI!一线大厂,做工强悍,支持1150针全系列CPU 。主板3个月包换,1年保修,七天包退!M-ATX规格,支持普通机箱。

mini pci-e支持无线网卡,支持半高。买一块AR9382,发射功率18dbm

注意,无须购买半高的AR9380,只有全高的才支持External Front End Module (FEM)

H81主板,淘宝上标注全新的,是展讯商家

主板到手,晚上LED灯光下看,全新,无水洗痕迹,有一股新板子的味道。板子上的序列号使用记号笔遮挡,但透过灯光可以看清。

此主板是 华硕台式机KADE的拆机板,价格要238元,贵在DC供电上。而且千兆网卡,是Intel的i217-v

总结,若非冲着e3 1220L v3的低功耗,以及这款板子的DC供电,它的价格还是太高了。

Intel® Xeon® Processor E3-1220L v3 规格

Intel® Xeon® Processor E3-1220L v3

4M Cache, 1.10 GHz

Essentials

Performance

Supplemental Information

Memory Specifications

Expansion Options

Package Specifications

Advanced Technologies

Security & Reliability