SEARCH

Enter your search query in the box above ^, or use the forum search tool.

You are not logged in.

#1 2014-01-23 14:13:05

OsmoHyttinen
Member
Registered: 2012-07-17
Posts: 31

UEFI settings and boot options?

I need help with this IOMMU thing...

Using the (newest?) #! 11 64bit,

Here it said "osmo:~$ uname -a" for some reason... sorry about that.

osmo@Fern:~$ uname -a
Linux Fern 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64 GNU/Linux

IOMMU disabled either from UEFI or with iommu=off
- boot takes less than 10 sec
- no USB ports work? (can't do anything with keyboard or mouse, haven't tried over SSH yet).

IOMMU enabled from UEFI, without setting any iommu= boot options
- boot takes a very long time as it "hangs" after printing 22 lines like:

[    1.676411] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.0 domain=0x0016 address=0x00000000be9fb880 flags=0x0010]

full dmesg here: http://54.213.154.174:777/B/recoverymodedmesg
- USB3 ports do not work even as USB2 ports
- reported CPU freq changes normally.


IOMMU on from the UEFI and a boot option iommu=soft or iommu=pt
- boot takes about 10 sec.
- reported CPU freq is "stuck at" 3500MHz
- USB3 ports work

Last edited by OsmoHyttinen (2014-01-24 12:12:30)

Offline

Help fund CrunchBang, donate to the project!

#2 2014-01-23 16:51:16

retf
#! CrunchBanger
From: On top of spaghetti
Registered: 2013-12-25
Posts: 200

Re: UEFI settings and boot options?

With 'IOMMU' off, you've turned off the memory mapping of the machine - which means nothing in the machine will be addressable.  So when setting

iommu=off

it's not surprising that nothing works...

After browsing through the posted dmesg log file, I noticed a 'swiotlb' reference.  This is what's enabled when you use:

iommu=soft

so, I assume the included dmesg is from a boot attempt with iommu set to soft.

'soft' means that a generic block of memory is set aside for assigning address space to devices within the system.  This usually works.

The page fault you receive when attempting to boot without any explicit options is probably related to a device that is attempting to access a memory location that is outside of the space designated for device addresses.

If the machine is a large 'regular' desktop machine, you could open up the case and remove all cards except for the video card - and see if it boots normally.  If you do this, make sure you turn off the PC AND unplug it, before taking the internals apart.

Also, you haven't indicated whether the machine has been running a GNU/Linux distro previously and just now has an issue, or if you are attempting to install #! - and ran into this problem, or have changed the hardware configuration and now things don't work as expected... Any details regarding the how and why you are doing would greatly assist in solving the issue...  I can guess that the machine is a working machine since the hostname has been assigned to 'osmo' --- however, without explicit statements, I'm just guessing as to the current state and problem.

Offline

#3 2014-01-23 22:02:15

OsmoHyttinen
Member
Registered: 2012-07-17
Posts: 31

Re: UEFI settings and boot options?

retf wrote:

Also, you haven't indicated whether the machine has been running a GNU/Linux distro previously and just now has an issue, or if you are attempting to install #! - and ran into this problem, or have changed the hardware configuration and now things don't work as expected...

I built the machine from separate, all new parts. No problems installing, though. I've been using this for about a month now.
It has only a WLAN/Bluetooth card besides the video card.
lspci http://54.213.154.174:777/B/lspci
lspci -v http://54.213.154.174:777/B/lspci%20-v

Last edited by OsmoHyttinen (2014-01-23 22:09:53)

Offline

#4 2014-01-24 00:06:11

retf
#! CrunchBanger
From: On top of spaghetti
Registered: 2013-12-25
Posts: 200

Re: UEFI settings and boot options?

From what you've now posted, it looks like the USB 3.0 controller is the problem.  Is there a way to disable the on-board USB 3.0 controller or set it to 'legacy' or something?  If there is a way to disable the USB 3.0, you could verify if the system boots without error with the USB 3.0 off.  Do have any USB 3.0 devices? If not and if the system boots normally without any boot options after disabling USB 3.0 - you should be fine...

Perhaps, there's a "Smart Auto", like some of the other forums indicate - which might load USB 3.0 drivers 'on-demand' when a USB 3.0 device is plugged in...

EDIT:  It might also be useful to determine which version of the XHCI kernel driver you are running...

sudo modinfo xhci_hcd

Last edited by retf (2014-01-24 00:52:27)

Offline

#5 2014-01-24 08:29:00

OsmoHyttinen
Member
Registered: 2012-07-17
Posts: 31

Re: UEFI settings and boot options?

osmo@Fern:~$ sudo modinfo xhci_hcd
filename:       /lib/modules/3.2.0-4-amd64/kernel/drivers/usb/host/xhci-hcd.ko
license:        GPL
author:         Sarah Sharp
description:    'eXtensible' Host Controller (xHC) Driver
alias:          pci:v*d*sv*sd*bc0Csc03i30*
depends:        usbcore
intree:         Y
vermagic:       3.2.0-4-amd64 SMP mod_unload modversions 
parm:           link_quirk:Don't clear the chain bit on a link TRB (int)

Offline

#6 2014-01-24 08:35:25

OsmoHyttinen
Member
Registered: 2012-07-17
Posts: 31

Re: UEFI settings and boot options?

retf wrote:

so, I assume the included dmesg is from a boot attempt with iommu set to soft.

No, the dmesg is from a successful, "recovery mode" boot with no iommu boot options, if set to soft, it does not print those

[    1.677255] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.0 domain=0x0016 address=0x00000000be9fb880 flags=0x0010]

lines at all and boots in about 10 sec (it takes tens of seconds when it prints, and "hangs" after the lines)

It "works the best" when booting without any iommu options, the USB3 ports just don't work at all, like I said (at least conky shows CPU freq changing, not the full 3500MHz when idle), the boot just takes very long, not an issue as I practically never reboot but those lines and that "hang" were something that caught my eye.

Here's the dmesg of the lastest boot 1.5 days ago http://54.213.154.174:777/B/dmesg%20iommu=pt
It was with the option iommu=pt (same as iommu=soft ?), took about 10 sec and all USB ports work. Conky reports CPU freq as constant 3500MHz despite the load.

Why does it take 10 seconds to boot? I got the same time (with the same SSD) with my laptop and this computer should be considerably more powerful.

"root partition read speed test"

osmo@Fern:~$ sudo dd if=/dev/sda3 of=/dev/null
149026816+0 records in
149026816+0 records out
76301729792 bytes (76 GB) copied, 205.037 s, 372 MB/s
osmo@Fern:~$ sudo dd if=/dev/sda3 of=/dev/null bs=4096
18628352+0 records in
18628352+0 records out
76301729792 bytes (76 GB) copied, 203.038 s, 376 MB/s

Last edited by OsmoHyttinen (2014-01-24 17:27:47)

Offline

#7 2014-01-24 13:41:04

retf
#! CrunchBanger
From: On top of spaghetti
Registered: 2013-12-25
Posts: 200

Re: UEFI settings and boot options?

As expected, allowing the OS to determine the memory map works.  You can see the xhci_hcd kernel driver loading and everything looks fine.

So far we have:
1. You can boot with everything functional by asking the memory mapper to allow the OS to set the mapping.
2. The BIOS/UEFI mapper fails to correctly locate the USB 3.0 controller, which results in a page fault - it's trying to set the address outside the allocated space.

I'm not sure what you mean by the CPU freq is stuck at 3500 MHz.  The information from both dmesg outputs is the same regarding CPU speed and available power states.

[   20.849209] powernow-k8: Found 1 AMD FX(tm)-8320 Eight-Core Processor            (8 cpu cores) (version 2.20.00)
[   20.849363] powernow-k8: Core Performance Boosting: on.
[   20.849505] powernow-k8:    0 : pstate 0 (3500 MHz)
[   20.849607] powernow-k8:    1 : pstate 4 (1400 MHz)

Maybe you could try updating the BIOS on the motherboard.  Perhaps the 'preferred' location for the VIA USB 3.0 controller was improperly set in the version the MB was shipped with...


***********************************************************************************************
EDIT --- below there's a more complete reasoning, if you are interested...     
***********************************************************************************************

This is important information from your dmesg log files:

[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009e800 (usable)
[    0.000000]  BIOS-e820: 000000000009e800 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000be861000 (usable)
[    0.000000]  BIOS-e820: 00000000be861000 - 00000000bea39000 (reserved)      <-----
[    0.000000]  BIOS-e820: 00000000bea39000 - 00000000bea41000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000bea41000 - 00000000bee33000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bee33000 - 00000000bf15a000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf15a000 - 00000000bf15b000 (usable)
[    0.000000]  BIOS-e820: 00000000bf15b000 - 00000000bf361000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf361000 - 00000000bf800000 (usable)
[    0.000000]  BIOS-e820: 00000000f8000000 - 00000000fc000000 (reserved)
[    0.000000]  BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
[    0.000000]  BIOS-e820: 00000000fec10000 - 00000000fec11000 (reserved)
[    0.000000]  BIOS-e820: 00000000fec20000 - 00000000fec21000 (reserved)
[    0.000000]  BIOS-e820: 00000000fed00000 - 00000000fed01000 (reserved)
[    0.000000]  BIOS-e820: 00000000fed61000 - 00000000fed71000 (reserved)
[    0.000000]  BIOS-e820: 00000000fed80000 - 00000000fed90000 (reserved)
[    0.000000]  BIOS-e820: 00000000fef00000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100001000 - 000000043f000000 (usable)

[    0.802597] PCI: Using ACPI for IRQ routing
[    0.808885] PCI: pci_cache_line_size set to 64 bytes
[    0.808990] reserve RAM buffer: 000000000009e800 - 000000000009ffff 
[    0.808992] reserve RAM buffer: 00000000be861000 - 00000000bfffffff   <-----
[    0.808994] reserve RAM buffer: 00000000bf15b000 - 00000000bfffffff 
[    0.808995] reserve RAM buffer: 00000000bf800000 - 00000000bfffffff 
[    0.808996] reserve RAM buffer: 000000043f000000 - 000000043fffffff 

[    0.780619] pci 0000:01:00.0: [1106:3483] type 0 class 0x000c03
[    0.780631] pci 0000:01:00.0: reg 10: [mem 0xfe500000-0xfe500fff 64bit]  <-----
[    0.780681] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.788060] pci 0000:00:09.0: PCI bridge to [bus 01-01]


[    1.676214] xhci_hcd 0000:01:00.0: setting latency timer to 64
[    1.676219] xhci_hcd 0000:01:00.0: xHCI Host Controller
[    1.676305] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 1
[    1.676411] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.0 domain=0x0016 address=0x00000000be9fb880 flags=0x0010]  <-----
[    1.676571] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.0 domain=0x0016 address=0x00000000be9fb880 flags=0x0010]
[    1.676710] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.0 domain=0x0016 address=0x00000000be9fb880 flags=0x0010]

The arrowed lines show the problem.  In the default booting scenario, the USB 3.0 controller is attempting to write or assign to an address that has been reserved by the BIOS for use by IRQs.  Several things are possible here, but it would appear that the motherboard chip set and BIOS have incompatible settings.  The problem is within the motherboard and not within the OS, as the OS is just 'following' the BIOS's mapping when no boot options are chosen.  As far as the precise issue with the motherboard, that's hard to say --- it could be that the interrupt vector at address '0xbe9fb880' doesn't exist or points to a location that would exist if you were - say - running MS Windows and the MS Windows driver had loaded some code that corresponds to some specialized function of the chip... it's really an unknown... and probably unknowable --- what the 'real' issue might be...

So, my suggestion is:
1. Try updating the BIOS.
2. Set the iommu to pt or soft during each boot.
3. Or make the iommu setting permanent by:

Saving this as /etc/modprobe.d/iommu-perm-setting.conf

## Added by me for iommu setting
options iommu=pt

Last edited by retf (2014-01-24 14:32:48)

Offline

#8 2014-01-24 17:49:12

OsmoHyttinen
Member
Registered: 2012-07-17
Posts: 31

Re: UEFI settings and boot options?

retf wrote:

I'm not sure what you mean by the CPU freq is stuck at 3500 MHz.  The information from both dmesg outputs is the same regarding CPU speed and available power states.

[   20.849209] powernow-k8: Found 1 AMD FX(tm)-8320 Eight-Core Processor            (8 cpu cores) (version 2.20.00)
[   20.849363] powernow-k8: Core Performance Boosting: on.
[   20.849505] powernow-k8:    0 : pstate 0 (3500 MHz)
[   20.849607] powernow-k8:    1 : pstate 4 (1400 MHz)

Maybe you could try updating the BIOS on the motherboard.  Perhaps the 'preferred' location for the VIA USB 3.0 controller was improperly set in the version the MB was shipped with...

So, my suggestion is:
1. Try updating the BIOS.
2. Set the iommu to pt or soft during each boot.
3. Or make the iommu setting permanent by:

Saving this as /etc/modprobe.d/iommu-perm-setting.conf

## Added by me for iommu setting
options iommu=pt
osmo@Fern:~$ lscpu 
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    2
Core(s) per socket:    4
Socket(s):             1
NUMA node(s):          1
Vendor ID:             AuthenticAMD
CPU family:            21
Model:                 2
Stepping:              0
CPU MHz:               3500.000
BogoMIPS:              7031.88
Virtualization:        AMD-V
L1d cache:             16K
L1i cache:             64K
L2 cache:              2048K
L3 cache:              8192K
NUMA node0 CPU(s):     0-7

There it says "Stepping: 0" and the frequency (at least seemingly) never changes.

Here's two lscpu examples from my laptop: idle and under load


And for the BIOS update, the MB should have the latest version that was available a month ago (I selected, and paid for a BIOS update when I ordered it)

I have already made it permanent by just adding the iommu=pt to /boot/grub/grub.cfg

menuentry 'CrunchBang GNU/Linux, with Linux 3.2.0-4-amd64' --class crunchbang --class gnu-linux --class gnu --class os {
	load_video
	insmod gzio
	insmod part_gpt
	insmod ext2
	set root='(hd0,gpt5)'
	search --no-floppy --fs-uuid --set=root ff874789-0a6b-4e1b-b0fd-48769be9e504
	echo	'Loading Linux 3.2.0-4-amd64 ...'
	linux	/boot/vmlinuz-3.2.0-4-amd64 root=UUID=ff874789-0a6b-4e1b-b0fd-48769be9e504 ro  quiet iommu=pt
	echo	'Loading initial ramdisk ...'
	initrd	/boot/initrd.img-3.2.0-4-amd64
}

Should it be "pt" or "soft" by the way? What's the difference between them?

Last edited by OsmoHyttinen (2014-01-24 18:07:07)

Offline

#9 2014-01-24 18:09:28

retf
#! CrunchBanger
From: On top of spaghetti
Registered: 2013-12-25
Posts: 200

Re: UEFI settings and boot options?

While adding your own options to the grub.cfg works, it's probably a better idea to separate it from the main file in order to keep better track of what you've added versus what's 'basic'... but it doesn't really matter unless you're a purist believer in pristine file organization...

Here's what the Linux kernel people have to say about the iommu settings:
http://pastebin.com/v28e6e9v

There's no reference here to 'pt' --- I've seen 'pt' listed elsewhere, so it seems to be a valid option - but my guess is it's the same as 'soft' --- In short, I'd use 'soft' rather than 'pt', but there seem to be tons of options to play with. Maybe you could try a few and mess with the BIOS settings a bit and viola! get a BIOS iommu configured system with a Linux kernel configured USB 3.0 driver...

Offline

Board footer

Powered by FluxBB

Copyright © 2012 CrunchBang Linux.
Proudly powered by Debian. Hosted by Linode.
Debian is a registered trademark of Software in the Public Interest, Inc.
Server: acrobat

Debian Logo