Project

General

Profile

Actions

Bug #392

open

coreboot 4.17 - SeaBIOS Windows 10 BSOD "ACPI Error"

Added by Pawel Radomychelski 3 months ago. Updated 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
board support
Target version:
Start date:
06/18/2022
Due date:
% Done:

0%

Estimated time:
Affected versions:
Needs backport to:
Affected hardware:
Lenovo ThinkPad X230 Tablet
Affected OS:
Windows 10

Description

Since CoreBoot 4.16 my Windows 10 cant boot from SeaBios, i get BSOD with "ACPI Error" very early.

Don't know which commit exactly brakes ACPI, but i can say, that in my CoreBoot 4.15 Image from 11/09/2021 Windows 10 is booting just fine from SeaBIOS.
Some time later under CoreBoot 4.16 i saw that windows is BSODing. Tried yesterday with CoreBoot 4.17 and its still broken.

I think, since CB4.16 there is something broken in ACPI. As i read, the problem is, that ACPI reserves some memory area, which in Windows is reserved for the system.

This [[[https://ticket.coreboot.org/issues/327]]] seems to be a similar problem, but the guy is using TianoCore instead of SeaBIOS.
Changing the line
OperationRegion (OPRG, SystemMemory, ASLS, 0x2000)
to
OperationRegion (OPRG, SystemMemory, ASLS, 0x1000)
doesnt fix it for SeaBios, but fix it for TianoCore.


Files

dmesg_cb415.txt (78.9 KB) dmesg_cb415.txt dmesg from CoreBoot 4.15 Pawel Radomychelski, 06/18/2022 09:04 PM
dmesg_cb417.txt (79.9 KB) dmesg_cb417.txt dmesg from CoreBoot 4.17 Pawel Radomychelski, 06/18/2022 09:04 PM
.config (19.7 KB) .config Coreboot .config Pawel Radomychelski, 06/18/2022 10:36 PM
CB4.17_BSOD.jpg (143 KB) CB4.17_BSOD.jpg Windows 10 BSDO ACPI Error Picture Pawel Radomychelski, 06/20/2022 04:47 PM
CB415.log (46.4 KB) CB415.log Coreboot 4.15 /sys/firmware/log Pawel Radomychelski, 06/22/2022 07:50 PM
CB417.log (128 KB) CB417.log Coreboot 4.17 /sys/firmware/log Pawel Radomychelski, 06/22/2022 07:50 PM
CB416.log (128 KB) CB416.log Coreboot 4.16 /sys/firmware/log Pawel Radomychelski, 06/27/2022 10:29 PM
dmesg_cb416.txt (81.8 KB) dmesg_cb416.txt dmesg from CoreBoot 4.16 Pawel Radomychelski, 06/27/2022 10:31 PM
Actions #1

Updated by Pawel Radomychelski 3 months ago

  • Related links updated (diff)
Actions #2

Updated by Pawel Radomychelski 3 months ago

CoreBoot 4.15

[    0.172669] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.172734] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.172796] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000cbfff window]
[    0.172871] pci_bus 0000:00: root bus resource [mem 0x000d4000-0x000fffff window]
[    0.172946] pci_bus 0000:00: root bus resource [mem 0x8ea00000-0xefffffff window]
[    0.173020] pci_bus 0000:00: root bus resource [mem 0xfed40000-0xfed44fff window]

CoreBoot 4.17

[    0.172744] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.172809] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.172872] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000cbfff window]
[    0.172948] pci_bus 0000:00: root bus resource [mem 0x000d4000-0x000fffff window]
[    0.173022] pci_bus 0000:00: root bus resource [mem 0x8ea00000-0xefffffff window]
[    0.173096] pci_bus 0000:00: root bus resource [mem 0x471600001-0xfffffffff window]
[    0.173170] pci_bus 0000:00: root bus resource [mem 0xfed40000-0xfed44fff window]

Maybe its the same bug like https://ticket.coreboot.org/issues/327,
but in Bug #327 Windows is starting in UEFI mode via Tianocore, in my bug Windows ist started in legacy mode via SeaBIOS.
Solution which is discussed in Bug #327 doesnt work for SeaBIOS.

Actions #3

Updated by Pawel Radomychelski 3 months ago

Actions #4

Updated by Pawel Radomychelski 3 months ago

  • Related links updated (diff)
  • Priority changed from High to Normal
Actions #5

Updated by Paul Menzel 3 months ago

  • Subject changed from Coreboot 4.16 & 4.17 - SeaBios Windows 10 BSOD "ACPI Error" to Coreboot 4.16 & 4.17 - SeaBIOS Windows 10 BSOD "ACPI Error"
Actions #6

Updated by Paul Menzel 3 months ago

  • Subject changed from Coreboot 4.16 & 4.17 - SeaBIOS Windows 10 BSOD "ACPI Error" to coreboot 4.16 & 4.17 - SeaBIOS Windows 10 BSOD "ACPI Error"
  1. Please attach a photo of the BSOD, or copy all messages from it.
  2. Please attach the coreboot log of both images. (If you do not have GNU/Linux installed, you can start it from a USB device with a GNU/Linux live image.
  3. If you can flash external to potentially recover, please bisect the issue.
Actions #7

Updated by Arthur Heymans 3 months ago

Actions #8

Updated by Pawel Radomychelski 3 months ago

Arthur Heymans wrote in #note-7:

Can you check if https://review.coreboot.org/c/coreboot/+/65250 fixes it?

@Arthur: This patch unfortunately doesn't fix it. :(

@Paul:

  1. Image of BSOD attached. No additional information.
  2. How can i create coreboot.log of both images?
  3. I can flash externaly and have also Debian 11 on another drive, so its no problem for me.
Actions #9

Updated by Paul Menzel 3 months ago

  1. Image of BSOD attached. No additional information.

Strange that there is no more specific error code.

  1. How can i create coreboot.log of both images?

If your Linux kernel is recent enough (maybe from backports), and has GOOGLE_MEMCONSOLE_COREBOOT selected

$ grep MEMCONSOLE /boot/config-5.18.0-2-amd64
CONFIG_GOOGLE_MEMCONSOLE=m
# CONFIG_GOOGLE_MEMCONSOLE_X86_LEGACY is not set
CONFIG_GOOGLE_MEMCONSOLE_COREBOOT=m

then

sudo modprobe memconsole-coreboot
sudo less /sys/firmware/log

Otherwise: In the coreboot source tree, make -C util/cbmem and sudo util/cbmem/cbmem -1.

Actions #10

Updated by Lance Zhao 3 months ago

If we can get a windows crash dump will be easier to debug the issue, we can use windbg to open generated dump file(memory.dmp most of the time). Then the detail info normally included,

Actions #11

Updated by Pawel Radomychelski 3 months ago

Lance Zhao wrote in #note-10:

If we can get a windows crash dump will be easier to debug the issue, we can use windbg to open generated dump file(memory.dmp most of the time). Then the detail info normally included,

I will try to create it, thanks

Actions #12

Updated by Pawel Radomychelski 3 months ago

Actions #13

Updated by Pawel Radomychelski 3 months ago

Paul Menzel wrote in #note-9:

  1. Image of BSOD attached. No additional information.

Strange that there is no more specific error code.

  1. How can i create coreboot.log of both images?

If your Linux kernel is recent enough (maybe from backports), and has GOOGLE_MEMCONSOLE_COREBOOT selected

$ grep MEMCONSOLE /boot/config-5.18.0-2-amd64
CONFIG_GOOGLE_MEMCONSOLE=m
# CONFIG_GOOGLE_MEMCONSOLE_X86_LEGACY is not set
CONFIG_GOOGLE_MEMCONSOLE_COREBOOT=m

then

sudo modprobe memconsole-coreboot
sudo less /sys/firmware/log

Otherwise: In the coreboot source tree, make -C util/cbmem and sudo util/cbmem/cbmem -1.

Attached both /sys/firmware/log from CoreBoot 4.15 and 4.17. Thanks

Actions #14

Updated by Pawel Radomychelski 3 months ago

  • File 062322-6125-01.dmp added
Actions #15

Updated by Pawel Radomychelski 3 months ago

Added Windows/Minidump *.dmp File.
Will also look how to upload MEMORY.dmp which is 1,4GB large.

Actions #16

Updated by Lance Zhao 3 months ago

Output of minidump shows that's not acpi bios error.
4: kd> !analyze -v


  • *
  • Bugcheck Analysis *
  • * *******************************************************************************

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: ffffdd80db3e1760, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffff8036dea1981, address which referenced memory

Debugging Details:

BUGCHECK_CODE: d1

BUGCHECK_P1: ffffdd80db3e1760

BUGCHECK_P2: 2

BUGCHECK_P3: 0

BUGCHECK_P4: fffff8036dea1981

READ_ADDRESS: Unable to get NonPagedPoolStart
Unable to get NonPagedPoolEnd
Unable to get PagedPoolStart
Unable to get PagedPoolEnd
ffffdd80db3e1760

CUSTOMER_CRASH_COUNT: 1

PROCESS_NAME: notmyfault64.exe

TRAP_FRAME: fffffe87853e24a0 -- (.trap 0xfffffe87853e24a0)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000000730055 rbx=0000000000000000 rcx=ffffdd80d0000340
rdx=0000000000000890 rsi=0000000000000000 rdi=0000000000000000
rip=fffff8036dea1981 rsp=fffffe87853e2630 rbp=0000000000000002
r8=ffffdd80e25e2da0 r9=0000000000000000 r10=ffffdd80d00002c0
r11=ffffdd80db3c1750 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na po nc
myfault+0x1981:
fffff8036dea1981 8b03 mov eax,dword ptr [rbx] ds:0000000000000000=????????
Resetting default scope

STACK_TEXT:

fffffe87853e2358 fffff80370609d69 : 000000000000000a ffffdd80db3e1760 0000000000000002 0000000000000000 : nt!KeBugCheckEx
fffffe87853e2360 fffff80370606069 : 0000000000000000 ffffca09b6b03740 0000000000000f4d 0000000000000000 : nt!KiBugCheckDispatch+0x69
fffffe87853e24a0 fffff8036dea1981 : ffffffffffffffff ffffca09af887080 fffffe87853e2730 0000000000000aa0 : nt!KiPageFault+0x469
fffffe87853e2630 ffffffffffffffff : ffffca09af887080 fffffe87853e2730 0000000000000aa0 0000000000000000 : myfault+0x1981
fffffe87853e2638 ffffca09af887080 : fffffe87853e2730 0000000000000aa0 0000000000000000 fffff8036dea1d3d : 0xffffffffffffffff
fffffe87
853e2640 fffffe87853e2730 : 0000000000000aa0 0000000000000000 fffff8036dea1d3d ffffdd8000730055 : 0xffffca09af887080
fffffe87853e2648 0000000000000aa0 : 0000000000000000 fffff8036dea1d3d ffffdd8000730055 00000000000f00ff : 0xfffffe87853e2730
fffffe87
853e2650 0000000000000000 : fffff8036dea1d3d ffffdd8000730055 00000000000f00ff 00000001`00000000 : 0xaa0

SYMBOL_NAME: myfault+1981

MODULE_NAME: myfault

IMAGE_NAME: myfault.sys

STACK_COMMAND: .cxr; .ecxr ; kb

BUCKET_ID_FUNC_OFFSET: 1981

FAILURE_BUCKET_ID: AV_myfault!unknown_function

OS_VERSION: 10.0.19041.1

BUILDLAB_STR: vb_release

OSPLATFORM_TYPE: x64

OSNAME: Windows 10

FAILURE_ID_HASH: {9745090a-9bce-ccba-c096-ca6e9ca04c64}

Followup: MachineOwner

Actions #17

Updated by Lance Zhao 3 months ago

To save time, you can install a windbg yourself and run !anlyze -v to get the detailed information. Then use !amli dl or other acpi debug provided by windbg itself.

Actions #18

Updated by Pawel Radomychelski 3 months ago

Sorry @Lance Zhao, but this is the wrong dump. It was created by the app, NotMyFault, which i used to play around with dumps.
It looks so, that the dump we need is not beeing created...

Actions #19

Updated by Pawel Radomychelski 3 months ago

  • File deleted (062322-6125-01.dmp)
Actions #20

Updated by Pawel Radomychelski 3 months ago

I could build Coreboot 4.16 with replacing the folder src/drivers/intel/gma/acpi from Coreboot 4.15.
With replaced acpi folder Windows 10 works without BSOD.
Unfortunately the same trick doesn't work for Coreboot 4.17. With 4.17 and acpi folder from 4.15 Windows 10 crashed very early with ACPI Error BSOD.
crash dump is also not created. Maybe because its crashes very early.

Actions #21

Updated by Pawel Radomychelski 3 months ago

  • File deleted (dmesg_cb416.txt)
Actions #23

Updated by Paul Menzel 3 months ago

Nice test. Please bisect with

$ git bisect start 4.16 4.15 -- src/drivers/intel/gma/acpi

and build and test each commit that is shown. (This only works, if the regression really is in src/drivers/intel/gma/acpi – otherwise, just use git bisect start 4.16 4.15.)

Actions #24

Updated by Pawel Radomychelski 3 months ago

Paul Menzel wrote in #note-23:

Nice test. Please bisect with

$ git bisect start 4.16 4.15 -- src/drivers/intel/gma/acpi

and build and test each commit that is shown. (This only works, if the regression really is in src/drivers/intel/gma/acpi – otherwise, just use git bisect start 4.16 4.15.)

Thanks for the info, but i am not very familiar with git commands.

When i am on the 4.16 branch (with default src/drivers/intel/gma/acpi) and i enter the command:

git bisect start 4.16 4.15 -- src/drivers/intel/gma/acpi
warning: refname '4.16' is ambiguous.
Switched to branch '4.16_branch'
Your branch is up to date with 'origin/4.16_branch'.
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[24eb605280b737ee63c72ede451bd5a7d0dce5e2] drivers/intel/gma/acpi: Replace Multiply(a,b) with ASL 2.0 syntax

I have tested all revisions and marked it all as good
I tested also the default 4.16 branch and windows still works.

So i would say, that the "bad" commit is somewhere between 4.16 and 4.17 branch and propably has nothing to do with src/drivers/intel/gma/acpi.

So need to bisect every commit between 4.17 and 4.16? OMG! :)

Actions #25

Updated by Pawel Radomychelski 3 months ago

  • Subject changed from coreboot 4.16 & 4.17 - SeaBIOS Windows 10 BSOD "ACPI Error" to coreboot 4.17 - SeaBIOS Windows 10 BSOD "ACPI Error"
  • Affected versions deleted (4.16)
Actions #26

Updated by Olexandr Nesterenko 2 months ago

I use t420 with Ivy Bridge CPU
Run bisect 4.16 - 4.17 and result:
# first bad commit: [f8daf86282c1be33643ace4e9f453e7548cab41c] nb/intel/sandybridge/acpi: Support setting PCI bars above 4G
After this commit i have BSOD ACPI Error.

Actions #27

Updated by Pawel Radomychelski 2 months ago

дуже дякую, Олексанре!

Thank you very much, Oleksandr for bisecting the problem!

hope the fix will be applied to the master branch!

Kind regards

Pawel Radomychelski

On July 24, 2022 at 9:19 AM Olexandr Nesterenko < coreboot@fe80.eu > wrote:

Issue #392 has been updated by Olexandr Nesterenko.

I use t420 with Ivy Bridge CPU
Run bisect 4.16 - 4.17 and result:
# first bad commit: [f8daf86282c1be33643ace4e9f453e7548cab41c] nb/intel/sandybridge/acpi: Support setting PCI bars above 4G
After this commit i have BSOD ACPI Error.


Bug #392: coreboot 4.17 - SeaBIOS Windows 10 BSOD "ACPI Error"
https://ticket.coreboot.org/issues/392#change-1048

  • Author: Pawel Radomychelski
  • Status: New
  • Priority: Normal
  • Category: board support
  • Target version: 4.17
  • Start date: 2022-06-18
  • Affected versions: 4.17, master
  • Related links: https://ticket.coreboot.org/issues/327
  • Affected hardware: Lenovo ThinkPad X230 Tablet

* Affected OS: Windows 10

Since CoreBoot 4.16 my Windows 10 cant boot from SeaBios, i get BSOD with "ACPI Error" very early.

Don't know which commit exactly brakes ACPI, but i can say, that in my CoreBoot 4.15 Image from 11/09/2021 Windows 10 is booting just fine from SeaBIOS.
Some time later under CoreBoot 4.16 i saw that windows is BSODing. Tried yesterday with CoreBoot 4.17 and its still broken.

I think, since CB4.16 there is something broken in ACPI. As i read, the problem is, that ACPI reserves some memory area, which in Windows is reserved for the system.

This [[ https://ticket.coreboot.org/issues/327 ]] seems to be a similar problem, but the guy is using TianoCore instead of SeaBIOS.
Changing the line
OperationRegion (OPRG, SystemMemory, ASLS, 0x2000)
to
OperationRegion (OPRG, SystemMemory, ASLS, 0x1000)
doesnt fix it for SeaBios, but fix it for TianoCore.

---Files--------------------------------
dmesg_cb415.txt (78.9 KB)
dmesg_cb417.txt (79.9 KB)
.config (19.7 KB)
CB4.17_BSOD.jpg (143 KB)
CB415.log (46.4 KB)
CB417.log (128 KB)
CB416.log (128 KB)
dmesg_cb416.txt (81.8 KB)

--
You have received this notification because you have either subscribed to it, or are involved in it.
To change your notification preferences, please click here: https://ticket.coreboot.org/my/account

Actions #28

Updated by Lance Zhao 2 months ago

Do we believe we need to have extra protection to check A4GB is not zero as
well?
Need to see the debug print about A4GB and A4GS at this point.

Lance

Pawel Radomychelski coreboot@fe80.eu 于2022年7月24日周日 17:00写道:

Issue #392 has been updated by Pawel Radomychelski.

дуже дякую, Олексанре!

Thank you very much, Oleksandr for bisecting the problem!

hope the fix will be applied to the master branch!

Kind regards

Pawel Radomychelski

On July 24, 2022 at 9:19 AM Olexandr Nesterenko < coreboot@fe80.eu > wrote:

Issue #392 has been updated by Olexandr Nesterenko.

I use t420 with Ivy Bridge CPU
Run bisect 4.16 - 4.17 and result:
# first bad commit: [f8daf86282c1be33643ace4e9f453e7548cab41c]
nb/intel/sandybridge/acpi: Support setting PCI bars above 4G

After this commit i have BSOD ACPI Error.


Bug #392: coreboot 4.17 - SeaBIOS Windows 10 BSOD "ACPI Error"

https://ticket.coreboot.org/issues/392#change-1048

  • Author: Pawel Radomychelski
  • Status: New
  • Priority: Normal
  • Category: board support
  • Target version: 4.17
  • Start date: 2022-06-18
  • Affected versions: 4.17, master
  • Related links: https://ticket.coreboot.org/issues/327
  • Affected hardware: Lenovo ThinkPad X230 Tablet

* Affected OS: Windows 10

Since CoreBoot 4.16 my Windows 10 cant boot from SeaBios, i get BSOD with
"ACPI Error" very early.

Don't know which commit exactly brakes ACPI, but i can say, that in my
CoreBoot 4.15 Image from 11/09/2021 Windows 10 is booting just fine from
SeaBIOS.
Some time later under CoreBoot 4.16 i saw that windows is BSODing. Tried
yesterday with CoreBoot 4.17 and its still broken.

I think, since CB4.16 there is something broken in ACPI. As i read, the
problem is, that ACPI reserves some memory area, which in Windows is
reserved for the system.

This [[
https://ticket.coreboot.org/issues/327

]] seems to be a similar problem, but the guy is using TianoCore instead of
SeaBIOS.
Changing the line
OperationRegion (OPRG, SystemMemory, ASLS, 0x2000)
to
OperationRegion (OPRG, SystemMemory, ASLS, 0x1000)
doesnt fix it for SeaBios, but fix it for TianoCore.

---Files--------------------------------
dmesg_cb415.txt (78.9 KB)
dmesg_cb417.txt (79.9 KB)
.config (19.7 KB)
CB4.17_BSOD.jpg (143 KB)
CB415.log (46.4 KB)
CB417.log (128 KB)
CB416.log (128 KB)
dmesg_cb416.txt (81.8 KB)

--
You have received this notification because you have either subscribed to
it, or are involved in it.
To change your notification preferences, please click here:
https://ticket.coreboot.org/my/account


Bug #392: coreboot 4.17 - SeaBIOS Windows 10 BSOD "ACPI Error"
https://ticket.coreboot.org/issues/392#change-1049

  • Author: Pawel Radomychelski
  • Status: New
  • Priority: Normal
  • Category: board support
  • Target version: 4.17
  • Start date: 2022-06-18
  • Affected versions: 4.17, master
  • Related links: https://ticket.coreboot.org/issues/327
  • Affected hardware: Lenovo ThinkPad X230 Tablet

* Affected OS: Windows 10

Since CoreBoot 4.16 my Windows 10 cant boot from SeaBios, i get BSOD with
"ACPI Error" very early.

Don't know which commit exactly brakes ACPI, but i can say, that in my
CoreBoot 4.15 Image from 11/09/2021 Windows 10 is booting just fine from
SeaBIOS.
Some time later under CoreBoot 4.16 i saw that windows is BSODing. Tried
yesterday with CoreBoot 4.17 and its still broken.

I think, since CB4.16 there is something broken in ACPI. As i read, the
problem is, that ACPI reserves some memory area, which in Windows is
reserved for the system.

This [[[https://ticket.coreboot.org/issues/327]]] seems to be a similar
problem, but the guy is using TianoCore instead of SeaBIOS.
Changing the line
OperationRegion (OPRG, SystemMemory, ASLS, 0x2000)
to
OperationRegion (OPRG, SystemMemory, ASLS, 0x1000)
doesnt fix it for SeaBios, but fix it for TianoCore.

---Files--------------------------------
dmesg_cb415.txt (78.9 KB)
dmesg_cb417.txt (79.9 KB)
.config (19.7 KB)
CB4.17_BSOD.jpg (143 KB)
CB415.log (46.4 KB)
CB417.log (128 KB)
CB416.log (128 KB)
dmesg_cb416.txt (81.8 KB)

--
You have received this notification because you have either subscribed to
it, or are involved in it.
To change your notification preferences, please click here:
https://ticket.coreboot.org/my/account


coreboot mailing list -- coreboot@coreboot.org
To unsubscribe send an email to coreboot-leave@coreboot.org

Actions

Also available in: Atom PDF