Opened 7 years ago

Closed 7 years ago

#332 closed incident (fixed)

LivingLab Down

Reported by: rick Owned by:
Keywords: Cc:
Location: Generiek

Description

De node reboot van #331 is niet succesvol gegaan, de node heeft een powercycle nodig.

Change History (7)

comment:1 Changed 7 years ago by huub

Probleem is dat bij livinglab een simpele reboot niet werkt: node blijft hangen. Powercycle werkt ook niet. Er moet iemand heen die op de serieele poort inlogt en 'reboot' typt.

comment:2 Changed 7 years ago by huub

Zie ticket #316, nog ter info de sessie op mijn netbook hieronder, misschien kan iemand daar nog wat verstandigs over zeggen.
~$ minicom -s

Welcome to minicom 2.5

OPTIONS: I18n
Compiled on May 2 2011, 00:39:27.
Port /dev/ttyUSB0

Press CTRL-A Z for help on special keys


OK ATS7=400L1 V X4 &c1Q
ATS7=400L1 not found
OK
OK reboot
Re�ڼ�o���������������뻀}��ݟo����m�mۀo������{�K��o��K���$ڤ�ҭ�y��P��7��{�Z_���� h
Consoles: serial port
BIOS drive C: is disk0
BIOS 640kB/261120kB available memory

FreeBSD/x86 bootstrap loader, Revision 1.1
(root@dell90, Tue Sep 25 21:42:12 CEST 2012)
Loading /boot/defaults/loader.conf
/boot/kernel/kernel data=0x619eec+0x35e10 syms=[0x4+0x68980+0x4+0x92d49]
\
Hit [Enter] to boot immediately, or any other key for command prompt.
Booting boot/kernel/kernel...
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994

The Regents of the University of California. All rights reserved.

FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-RELEASE #0: Tue Sep 25 22:44:22 CEST 2012

root@dell90:/usr/obj/nanobsd.wleiden-hybrid/usr/src/sys/kernel.wleiden i386

CPU: Geode(TM) Integrated Processor by AMD PCS (498.06-MHz 586-class CPU)

Origin = "AuthenticAMD" Id = 0x5a2 Family = 5 Model = a Stepping = 2
Features=0x88a93d<FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CLFLUSH,MMX>
AMD Features=0xc0400000<MMX+,3DNow!+,3DNow!>

real memory = 268435456 (256 MB)
avail memory = 253112320 (241 MB)
pnpbios: Bad PnP BIOS data checksum
K6-family MTRR support enabled (2 registers)
cryptosoft0: <software crypto> on motherboard
pcib0: <Host to PCI bridge> pcibus 0 on motherboard
pci0: <PCI bus> on pcib0
Geode LX: PC Engines ALIX.2 v0.99h tinyBIOS V1.4a (C)1997-2007
glxsb0: <AMD Geode LX Security Block (AES-128-CBC, RNG)> mem 0xefff4000-0xefff70
vr0: <VIA VT6105M Rhine III 10/100BaseTX> port 0x1000-0x10ff mem 0xe0000000-0xe0
vr0: Quirks: 0x2
vr0: Revision: 0x96
miibus0: <MII bus> on vr0
ukphy0: <Generic IEEE 802.3u media interface> PHY 1 on miibus0
ukphy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vr0: Ethernet address: 00:0d:b9:29:4d:50
vr1: <VIA VT6105M Rhine III 10/100BaseTX> port 0x1400-0x14ff mem 0xe0040000-0xe0
vr1: Quirks: 0x2
vr1: Revision: 0x96
miibus1: <MII bus> on vr1
ukphy1: <Generic IEEE 802.3u media interface> PHY 1 on miibus1
ukphy1: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vr1: Ethernet address: 00:0d:b9:29:4d:51
vr2: <VIA VT6105M Rhine III 10/100BaseTX> port 0x1800-0x18ff mem 0xe0080000-0xe0
vr2: Quirks: 0x2
vr2: Revision: 0x96
miibus2: <MII bus> on vr2
ukphy2: <Generic IEEE 802.3u media interface> PHY 1 on miibus2
ukphy2: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vr2: Ethernet address: 00:0d:b9:29:4d:52
ath0: <Atheros 5212> mem 0xe00c0000-0xe00cffff irq 9 at device 12.0 on pci0
ath0: AR5212 mac 5.9 RF5112 phy 4.3
isab0: <PCI-ISA bridge> port 0x6000-0x6007,0x6100-0x61ff,0x6200-0x623f,0x9d00-00
isa0: <ISA bus> on isab0
atapci0: <AMD CS5536 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x30
ata0: <ATA channel 0> on atapci0
ata1: <ATA channel 1> on atapci0
ohci0: <OHCI (generic) USB controller> mem 0xefffe000-0xefffefff irq 12 at devi0
usbus0: <OHCI (generic) USB controller> on ohci0
ehci0: <AMD CS5536 (Geode) USB 2.0 controller> mem 0xefffd000-0xefffdfff irq 120
usbus1: EHCI version 1.0
usbus1: <AMD CS5536 (Geode) USB 2.0 controller> on ehci0
cpu0 on motherboard
pmtimer0 on isa0
orm0: <ISA Option ROM> at iomem 0xe0000-0xea7ff pnpid ORM0000 on isa0
atrtc0: <AT realtime clock> at port 0x70 irq 8 on isa0
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> at port 0x40 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
uart0: <16550 or compatible> at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
uart0: console (9600,n,8,1)
uart1: <16550 or compatible> at port 0x2f8-0x2ff irq 3 on isa0
Timecounters tick every 4.000 msec
IPsec: Initialized Security Association Processing.
ipfw2 (+ipv6) initialized, divert enabled, nat loadable, rule-based forwarding d
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 480Mbps High Speed USB v2.0
ugen0.1: <AMD> at usbus0
uhub0: <AMD OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <AMD> at usbus1
uhub1: <AMD EHCI root HUBstray irq7
, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ada0 at ata0 bus 0 scbus0 target 0 lun 0
ada0: <CF 1GB 20110221> ATA-7 device
ada0: 100.000MB/s transfers (UDMA5, PIO 512bytes)
ada0: 967MB (1981728 512 byte sectors: 16H 63S/T 1966C)
ada0: Previously was knownuhub0: 4 ports with 4 removable, self powered

as ad0

Timecounter "TSC" frequency 498062834 Hz quality 800
GEOM: ada0s1: geometry does not match label (32h,63s != 16h,63s).
GEOM: ada0s2: geometry does not match label (32h,63s != 16h,63s).
Root mount waiting for: usbus1
Root mount waiting for: usbus1
uhub1: 4 ports with 4 removable, self powered
Root mount waiting for: usbus1
ugen1.2: <vendor 0x0b95> at usbus1
axe0: <vendor 0x0b95 product 0x772b, rev 2.00/0.01, addr 2> on usbus1
Trying to mount root from ufs:/dev/ada0s1a [ro]...
miibus3: <MII bus> on axe0
ukphy3: <Generic IEEE 802.3u media interface> PHY 16 on miibus3
ukphy3: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
ue0: <USB Ethernet> on axe0
ue0: Ethernet address: 00:60:6e:01:0c:53
Setting hostuuid: 79e6163b-bfde-11d3-8d41-000db9294d50.
Setting hostid: 0x970ac636.
Entropy harvesting: interrupts ethernet point_to_point kickstart.
Starting file system checks:
/dev/ada0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/ada0s1a: clean, 158560 free (3008 frags, 19444 blocks, 0.4% fragmentation)

/dev/ada0s3: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/ada0s3: clean, 9559 free (39 frags, 1190 blocks, 0.4% fragmentation)

Mounting local file systems:.
Setting hostname: HybridLivingLab?.wleiden.net..
vr0: link state changed to UP
DHCPDISCOVER on vr0 to 255.255.255.255 port 67 interval 8
DHCPDISCOVER on vr0 to 255.255.255.255 port 67 interval 8
DHCPOFFER from 192.168.1.1
DHCPOFFER from 192.168.1.1
DHCPOFFER already seen.
DHCPREQUEST on vr0 to 255.255.255.255 port 67
DHCPACK from 192.168.1.1
bound to 192.168.1.106 -- renewal in 43200 seconds.
vr1: link state changed to UP
vr2: link state changed to UP
wlan0: Ethernet address: 90:a4:de:b7:ae:76
ue0: link state changed to DOWN
Starting Network: lo0 vr0 vr1 vr2 ath0 pflog0 ipfw0 enc0 ue0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384

options=3<RXCSUM,TXCSUM>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9
inet 127.0.0.1 netmask 0xff000000
inet 172.31.255.1 netmask 0xffffffff
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

vr0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

options=8280b<RXCSUM,TXCSUM,VLAN_MTU,WOL_UCAST,WOL_MAGIC,LINKSTATE>
ether 00:0d:b9:29:4d:50
inet 192.168.1.106 netmask 0xffffff00 broadcast 192.168.1.255
inet 192.168.1.100 netmask 0xffffff00 broadcast 192.168.1.255
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active

vr1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

options=8280b<RXCSUM,TXCSUM,VLAN_MTU,WOL_UCAST,WOL_MAGIC,LINKSTATE>
ether 00:0d:b9:29:4d:51
inet 172.16.17.105 netmask 0xfffffff8 broadcast 172.16.17.111
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active

vr2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

options=8280b<RXCSUM,TXCSUM,VLAN_MTU,WOL_UCAST,WOL_MAGIC,LINKSTATE>
ether 00:0d:b9:29:4d:52
inet 172.16.17.113 netmask 0xfffffff8 broadcast 172.16.17.119
inet 172.16.4.172 netmask 0xfffffff8 broadcast 172.16.4.175
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active

ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 2290

ether 90:a4:de:b7:ae:76
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: IEEE 802.11 Wireless Ethernet autoselect mode 11g <hostap>
status: running

pflog0: flags=0<> metric 0 mtu 33200

nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536

nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

enc0: flags=0<> metric 0 mtu 1536

nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

options=80000<LINKSTATE>
ether 00:60:6e:01:0c:53
inet 172.16.4.60 netmask 0xfffffff8 broadcast 172.16.4.63
inet 172.16.5.5 netmask 0xfffffff8 broadcast 172.16.5.7
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (none)
status: no carrier

Starting devd.
Starting Network: usbus0.
Starting Network: usbus1.
Starting Network: pflog0.
pflog0: flags=0<> metric 0 mtu 33200

nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Starting Network: enc0.
enc0: flags=0<> metric 0 mtu 1536

nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

ue0: listray irq7
nk state changed to UP
Enabling pf.
route: writing to routing socket: File exists
add net default: gateway 192.168.1.1: route already in table
add net 172.16.0.0: gateway 127.0.0.1
Additional inet routing options: gateway=YES.
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
add net fe80::: gateway ::1
add net ff02::: gateway ::1
Generating host.conf.
Starting lvrouted.
Waiting for 8.8.8.8 to respond to ICMP.
ELF ldconfig path: /lib /usr/lib /usr/local/lib
a.out ldconfig path: /usr/lib/aout
Creating and/or trimming log files.
Starting syslogd.
Jan 1 00:01:07 HybridLivingLab? syslogd: sendto: Network is unreachable
wrote key file "/etc/namedb/rndc.key"
Starting named.
Setting date via ntp.
Clearing /tmp (X related).
Starting http302.
Updating motd:.
Starting ntpd.
Starting dhcpd.
Starting tinyproxy.
Starting thttpd.
Starting snmpd.
Starting nrpe2.
Starting sshd.
Starting watchdogd.
Starting cron.
Starting background file system checks in 60 seconds.
May 27 14:25:32 HybridLivingLab? ntpd[1974]: sendto(213.136.0.252) (fd=22): No rt
May 27 14:25:32 HybridLivingLab? ntpd[1974]: sendto(81.171.44.131) (fd=22): No rt

Mon May 27 14:25:32 UTC 2013

FreeBSD/i386 (HybridLivingLab?.wleiden.net.) (ttyu0)

login:

comment:3 Changed 7 years ago by huub

In de config stond zowel een statisch ip adres als dhcp-client. De interface kreeg twee ip-adressen. Ik heb de dhcp-client er af gehaald en de interface statisch op .100 gezet. Dat is een tijdje goed gegaan.

comment:4 in reply to:  1 Changed 7 years ago by rick

Replying to huub:

Probleem is dat bij livinglab een simpele reboot niet werkt: node blijft hangen. Powercycle werkt ook niet. Er moet iemand heen die op de serieele poort inlogt en 'reboot' typt.

Dat is knap onhandig, Ik zal eens kijken of ik een extra waarschuwing in de MOTD kan bakken, dit soort dingen zijn niet te weten als je remote ingelogd bent.

Voor de issue te fixen: bordje + CF vervangen lijkt mij.

comment:5 Changed 7 years ago by huub

Resolution: fixed
Status: newclosed

Rene en ik zijn met laddertje en reservespullen langs geweest. Gelukkig iets gevonden dat niet OK was: een los draadje in het kroonsteentje boven de galerijdeur (!). Meteen de nodeconfiguratie gewijzigd ivm het opheffen van de lokale internetverbinding (zie genesis).

comment:6 Changed 7 years ago by rene

Resolution: fixed
Status: closedreopened

Node is alweer enige tijd down.

Harde reboot op 12-10-2013 heeft niet geholpen.

comment:7 Changed 7 years ago by huub

Resolution: fixed
Status: reopenedclosed

zachte reboot wel. Meteen de config ververst, dwz weer in overeenstemming gebracht met genesis.

Note: See TracTickets for help on using tickets.