Created attachment 304135 [details]
dmesg output

grep -A 40 PS2K dsdt.dsl | grep IRQ -A 1

gives me the output:

                IRQNoFlags ()
                    {1}

Created attachment 304136 [details]
acpi.log

The ASL for the PS2K device:

```
    Scope (_SB.PCI0)
    {
        Device (PS2K)
        {
            Name (_HID, "MSFT0003")  // _HID: Hardware ID
            Name (_CID, EisaId ("PNP0303") /* IBM Enhanced Keyboard (101/1
02-key, PS/2 Mouse) */)  // _CID: Compatible ID
            Name (_UID, Zero)  // _UID: Unique ID
            Method (_STA, 0, NotSerialized)  // _STA: Status
            {
                Return (0x0F)
            }

            Name (_CRS, ResourceTemplate ()  // _CRS: Current Resource Set
tings
            {
                IO (Decode16,
                    0x0060,             // Range Minimum
                    0x0060,             // Range Maximum
                    0x00,               // Alignment
                    0x01,               // Length
                    )
                IO (Decode16,
                    0x0064,             // Range Minimum
                    0x0064,             // Range Maximum
                    0x00,               // Alignment
                    0x01,               // Length
                    )
                IRQNoFlags ()
                    {1}
            })
            Name (_PRS, ResourceTemplate ()  // _PRS: Possible Resource Settings
            {
                StartDependentFn (0x00, 0x00)
                {
                    IO (Decode16,
                        0x0060,             // Range Minimum
                        0x0060,             // Range Maximum
                        0x00,               // Alignment
                        0x01,               // Length
                        )
                    IO (Decode16,
                        0x0064,             // Range Minimum
                        0x0064,             // Range Maximum
                        0x00,               // Alignment
                        0x01,               // Length
                        )
                    IRQNoFlags ()
                        {1}
                }
                EndDependentFn ()
            })
            Method (_PSW, 1, NotSerialized)  // _PSW: Power State Wake
            {
                KBFG = Arg0
            }
        }

        Scope (\)
        {
            Name (KBFG, One)
        }
    }
```

Tamin commented in https://bugzilla.kernel.org/show_bug.cgi?id=216158#c83 that it might be a regression by commit 9946e39fe8d0 (ACPI: resource: skip IRQ override on AMD Zen platforms) [1], which was added in Linux 6.0-rc1.

Could you please test Linux 5.20 [2]?

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9946e39fe8d0a5da9eb947d8e40a7ef204ba016e
[2]: https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.19/

@Tamin,

actually have the notebook now for a month and tried to get this running
initially with ubuntu 22.04 lts and 18.04 lts.
earliest kernel i tried was 5.18, but it neither worked under this one.
That was why i switched back and hoped for kernel 6.2 to fix my issue.

Just for the record, Linux 6.2.6 logs:

```
[    0.000000] Linux version 6.2.0-20-generic (buildd@lcy02-amd64-035) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.2.0-17ubuntu1) 12.2.0, GNU ld (GNU Binutils for Ubuntu) 2.40) #20-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr  6 07:48:48 UTC 2023 (Ubuntu 6.2.0-20.20-generic 6.2.6)
[…]
[    0.000000] DMI: ASUSTeK COMPUTER INC. ASUS TUF Gaming A16 FA617NS_FA617NS/FA617NS, BIOS FA617NS.306 03/07/2023
[…]
[    0.437446] i8042: PNP: PS/2 Controller [PNP0303:PS2K] at 0x60,0x64 irq 1
[    0.437448] i8042: PNP: PS/2 appears to have AUX port disabled, if this is incorrect please boot with i8042.nopnp
[    0.438665] serio: i8042 KBD port at 0x60,0x64 irq 1
[    0.438721] mousedev: PS/2 mouse device common for all mice
```

The touchpad, mute microfon button and display brightness controls actually work.
But all other keys don't. 
Should i run kernel 5.19 again and give the outputs again?

i8042.nopnp
i actually modified this earlier, i think under 22.04, but without any effect.

Sorry i mean i modified grub to exclude from boot. sorry english is not my first language

Created attachment 304142 [details]
dmesg 5.19 output

@Paul
just reinstalled 22.04 and back to kernel 5.19, demsg output is above. Behavior is the same.

grep -A 40 PS2K dsdt.dsl | grep IRQ -A 1
                IRQNoFlags ()
                    {1}

Created attachment 304143 [details]
dmesg grep acpi

Created attachment 304144 [details]
acpi.log kernel 5.19

@Paul just insalled the kernel from your link
now there is some response on the keyboard, but its uncontrollable (just repeating some letters as h continously and can't type anything anymore)

Created attachment 304145 [details]
Reenable IRQ override on Asus TUF FA617NS

@sebj: Can you try applying the following patch to 6.2.11+ kernel? It basically re-enables he override that https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9946e39fe8d0a5da9eb947d8e40a7ef204ba016e disabled. 

I took some educated guesses as to the dmi_system_id and irq_override_cmp structure based on your dmesg posts. However, if this doesn't work we can confirm the values to make sure the reason its not working is not just a typo in my patch.

Tamim

Thanks Tamim,

Will check asap. Just went oustide, but will provide as soon as I'm nack.

Thanks so much for your all support

No problem. 

Also just so everyone knows, the disable IRQ override patch was unfortunately back ported to both 5.19 [1] and 5.15 [2]. Also based on the package source [3] for the kernel seb.j tried I think that 5.19 kernel might have had it too. As a result, if this is indeed the problem we are going to need to apply a similar patch to those kernels or use an older version of them that doesn't have the patch applied

Tamim 


[1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/drivers/acpi/resource.c?h=linux-5.19.y&id=37c81d9f1d1b1458894454efcb857f6a769b6bc4
[2] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/drivers/acpi/resource.c?h=v5.15.107&id=3bb12efc5e4d2b237230a4c6919b6fcf81d61190
[3] https://bugs.launchpad.net/ubuntu/+source/linux/5.19.0-38.39

(In reply to seb.j from comment #12)
> @Paul
> just reinstalled 22.04 and back to kernel 5.19, demsg output is above.
> Behavior is the same.

If you want to avoid the rebuild to try Tamim’s patch, please use 5.19 (.0) proper. The test you did with 22.04, used Linux 5.19.17, which had the problematic patch backported. (Big thanks to Tamim for spotting that.)

    [    0.000000] Linux version 5.19.0-38-generic (buildd@lcy02-amd64-001) (x86_64-linux-gnu-gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #39~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Mar 17 21:16:15 UTC 2 (Ubuntu 5.19.0-38.39~22.04.1-generic 5.19.17)

But, if you know how to build a Linux kernel [1], trying Tamim’s patch is easiest.


[1]: https://bugzilla.kernel.org/show_bug.cgi?id=217323#c5

@paul 
tried the patch again, but rn into some issue, that i couldn't identify

Created attachment 304146 [details]
git error

I now try to patch the kernal with tamim's patch and it's running now (after a few tries without the right environment).
It's the first time, that i compile a kernel... I'm normally just a normal user, so please forgive me in case i do some stupid error here, I will follow your explanations as good as i can

(In reply to seb.j from comment #21)
> @paul 
> tried the patch again, but run into some issue, that i couldn't identify

Ah, sorry for not explaining further:

The `wget` line downloads the patch for the problematic device in the bug 217323. Instead download and save Tamim’s patch with the browser:

    $ git clone https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
    $ cd linux
    $ # download and save Tamim’s patch
    $ patch -p1 < /path/to/the/patch/from/tamim/asus-tuf-6.2.patch
    $ git commit -a -m "Re-enable IRQ for Asus Tuf,
    $ make localmodconfig
    $ make bindeb-pkg # install missing program X with `sudo apt install X`
    $ sudo dpkg -i ../linux-image-*.deb

Hopefully that works.

Hey Paul,

Thanks so much for the additional help and explanation.
Will try that asap (currenlty outside again).
I don't want to trash to much here in the thread since i'm total new to all of this.

Actually, when i build the kernel with patch, the config making worked for me, but when i apply it with Paul's method, I get questions for the config

  64bit/32bit division and modulo test (TEST_DIV64) [N/m/y/?] n

sorry to ask, but where do i get the infos for this?

I think the default value here is fine. That would be the option with the capital letter (ie: `N` in this case) or just pressing enter will select it too. The reason this is happening is that `make localmodconfig` tries to your current kernel compilation to fill in the values for as much of these as possible but if a newer kernel introduces a new config you will need to still specify a value for it.

For any of the kernels you have compiled with the patch, did a reboot into them fix your keyboard?

Thanks,
Tamim

Hey Tamim,

Somehow,
I installed accroding to Paul's instructions, while compiling of the kernel by myself actually took very long and wasn't finished, when i came back.

My keyboard responds now, but in a very strange way.
Gives wrong key outputs, freezes sometimes and if i press keys, i will stuck on some letter and repeat it almost endlessly (until pressing oter keys, han i will repeat other letters, but all are the wrong ones, (n is like r etc).
i have the normal std. us layout in settings.
should i give another dmesg output?

kernel is now
6.3.0-rc6+

Created attachment 304147 [details]
patched dmesg output

In the past, I had actually already tried your older patch for the expert book and the behaviour was very similar.

@seb.j: I looked into your dmesg and it does look like its now overriding the IRQ now, so there doesn't seem to be a typo in the patch:

[    0.315700] ACPI: IRQ 1 override to edge, low(!)

However, your keyboard is not working so I think there is another problem. Can you try reverting the patch and running the stock 6.3.0-rc6 kernel and let us know how the keyboard behaves? You can do that by:

    $ cd linux # go to directory where you compiled the kernel previously
    $ patch -R -p1 < /path/to/the/patch/from/tamim/asus-tuf-6.2.patch
    $ git commit -a -m "Revert IRQ override for Asus TUF"
    $ make localmodconfig
    $ make bindeb-pkg # install missing program X with `sudo apt install X`
    $ sudo dpkg -i ../linux-image-*.deb

I am just trying to see if our patch made the keyboard at least register input and we need to change other things or if the patch didn't help at all.

Thanks,
Tamim

Also for the expertbook patch you tried, which kernel did you do that on? The same we are testing with now (ie: 6.3.0-rc6+)?

Tamim

Hey Tamim,

Just running the unpatching now.
I think the expert patch i've tried on the kernel delivered with 22.04 (5.19).

BTW, thanks for all the help here

With the patch applied, there seems to be feedback on he keyboard, but also a lot of ghost typing.
Had to go back to 6.2 to be able to work on the laptop again.

So basically the same with the patch reverted? If possible could you post the dmesg of the stock kernel too for comparison purposes?

Tamim

Created attachment 304148 [details]
dmesg 6.3 patch removed

You mean this one right?

Keyboard is now again fully unresponsive

Created attachment 304149 [details]
kernel 6.2.11 no patch

I was reading the original thread [1] where the Ryzen patch was submitted and it seems like with the patch I proposed above we are basically back to the old behavior (ie: ghost typing, keyboard typing delay etc). So unfortunately even though our patch is making the keyboard do "something" I don't think its helping.

Can you try adding the kernel command line flag i8042.debug to the kernel command line on the unpatched 6.3 kernel and attach the dmesg? Instructions on doing that in ubuntu can be found on https://wiki.ubuntu.com/Kernel/KernelBootParameters. Feel free to add either temporarily or permanently. Hopefully with that we will have clues as to what to look at next since it looks like my original idea about the regression is incorrect.

Tamim 

[1] https://bbs.archlinux.org/viewtopic.php?id=277260

Created attachment 304150 [details]
added debug to grub

Hey Tamim,

Added the debug and dmesg is above

Created attachment 304151 [details]
Test patch to force IRQ 1 to level high

@seb.j: Comparing your i8042 debug output to https://bbs.archlinux.org/viewtopic.php?pid=2041563#p2041563 it seems like your keyboard is behaving pretty much identically to other Ryzen laptops before the IRQ override disabling patch was added even though this patch is in the 6.3 kernel you are running.

Due to that, I suspect the problem is still IRQ setup related. As a result, taking inspiration from https://bugzilla.kernel.org/show_bug.cgi?id=213031#c13, I have attached two patches, one that sets the IRQ up as level_high and another that sets it up as level_low to see if either of these options gets your keyboard to work. Unfortunately, we wont be able to merge these into the kernel but if either works we can hopefully adapt them to something more generic that can be merged. 

In order to apply them, you should be able to use the instructions mentioned previously [1] expect without the `git clone`. If the first patch doesn't work though make sure to revert it though with `patch -R -p1 < /path/to/first/patch` before you do the `patch -p1 < /patch/to/second/patch`.

Let me know if you have any questions.

Tamim

[1] https://bugzilla.kernel.org/show_bug.cgi?id=217336#c24

Created attachment 304152 [details]
Test patch to force IRQ 1 to level low

Hey Tamim,

Thanks for this.
Running the patching now, but will be out for work again soon. Will let you know tonight, if one of those did it.

Created attachment 304153 [details]
kernel_high_patch_dmesg_output

high patch output, keyboard isn't responisve
will test the low patch now

Created attachment 304154 [details]
dmesg 6.3 patch lo

ddddddddddddddddddddddddddddddddddddddddddddddddeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeesssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhgggggggggggggggggggggggggggggggggggggggggggwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww

sorry for above, but this is what i got in ghost inputs, after opening libre office. interestingly it also disabled some keys on my usb connected keyboard (t and w, i saw), wiht ht elow patch applied. overall it actually behaves very similar to the ryzen patch

Adding Hui Wang to Cc:. Hui, this problem is happening also with Ubuntu GNU/Linux.

Unfortunately, at this point I don't really have any good ideas. We have tried all 4 setup options and none of them gotten us anything that works:

* EDGE_HIGH: Default unpatched kernel. Keyboard was unresponsive
* EDGE_LOW: First patch to re-enable override. Keyboard producing ghost typing
* LEVEL_HIGH: test-asus-tuf-level-high.patch. Keyboard was unresponsive
* LEVEL_LOW: test-asus-tuf-level-low.patch. Keyboard producing ghost typing

Interestingly, based on the i8042.debug dmesg output it looks like the LOW modes have a more complete i8042 setup but a lot of ghost typing when compared to the HIGH modes where it seems like it just hangs. We could try the low mode (ie: the originally proposed patch) and some i8042 options like `i8042.dumbkbd`, `i8042.noaux` and `i8042.nomux` independently and see if that does anything. Not sure if that will help anything though.

However, I am hoping @Hui Wang has some better ideas as to what to try or look into since he was the originally the one that patched this issue on the original mideon laptop that was impacted.

Tamim

Thanks for all the help Tamim and Paul,

At least we got something. I will get v
back to request asus one more time, to support on this. Actually they had advertised linux compatibility  in my region for this model (that was actually the selling point to me).
Maybe they can give at least some hint, what we have to improve here.

One more thing, does the keyboard work correctly with a non-GNU/Linux like *BSD or Microsoft Windows?

Dear Paul,

Yes, the laptop actually arrived with Windows 11 and everything worked fine.
The problem only occurs ubder Ubuntu.
Other distro's of ubuntu, actually never had time to reallyy test this.

Is here eventual a specific question, I should ask Asus, to helpt here forward?
Eventually we will need details on IRQ settings for the keyboard, am I correct here?

Sorry, no idea. Did ASUS already tell you what system firmware version and GNU/Linux distribution they tested with?

It might also be a good idea to create an issue in Canonical’s/Ubuntu’s issue/bug tracker Launchpad, and set the metadata there to this upstream report.

Actually didn't ask this initially.
Thanks for the hints and will let you know if there is any update

Hey guys,

Just got an update by asus and amd, that they will not support on the issue....

But on the other hand, asus released a new bios update (308) and after i updated and tried the asus patch on kernel 6.3 again and see, there is some movement.

Initially, go he ghost typing issue again, i thn activated "slow keys" "sticky keys" and "bouncy keys" from the accessibility menu (i always did his earleier too, but without any good results)

but this time, i was able to type my password and login wiht enter, even got number response form the numpad (tried in libre office after start).

the input will be hardly delayed and it hangs after a while, bt i'm amazd it is working somehow.

does this give you more info? will attach a new dmesg, when back form work today

Created attachment 304198 [details]
bios 308 dmesg asus patch applied

could it eventually be, that the acpi usb hub config error is connected to this?
i assum the keyboard is connected on that bus.

Wow, yah that bios must have changed something with their setup of the keyboard. The kernels communication with the keyboard as outputted by `i8042.debug` actually has the keyboard responding back rather then just ignoring the kernel (you can see it too by downloading your dmesg attachment and then running `grep i8042 path_to_dmesg_file.txt`). At this point, since you are much closer to something working it would be good to try the kernel options `i8042.dumbkbd`, `i8042.noaux` and `i8042.nomux` individually in various combinations to see if we can get it to work reliably. Hopefully, that gives as idea about what is really broken but if not at least you have an option to get your keyboard working for now.

> could it eventually be, that the acpi usb hub config error is connected to
> this?
> i assum the keyboard is connected on that bus.

Unfortunately, I think if the i8042 driver is detecting it then it should be using the PS/2 bus and not USB. However, if the keyboard is meant to use USB but is being picked up by i8042 then that would definitely explain why its behaving so badly. 

One other thing that would be good to look at as well just in case the bios changed it is the the value of PS2K in the DSDT table. You can obtain that doing: 

cat /sys/firmware/acpi/tables/DSDT > dsdt.dat
iasl -d dsdt.dat
grep -A 40 PS2K dsdt.dsl

Thanks,
Tamim

Hey Tamim, 

Thanks for the help again.

output of the commands as below:



sudo grep -A 40 PS2K dsdt.dsl
        Device (PS2K)
        {
            Name (_HID, "MSFT0003")  // _HID: Hardware ID
            Name (_CID, EisaId ("PNP0303") /* IBM Enhanced Keyboard (101/102-key, PS/2 Mouse) */)  // _CID: Compatible ID
            Name (_UID, Zero)  // _UID: Unique ID
            Method (_STA, 0, NotSerialized)  // _STA: Status
            {
                Return (0x0F)
            }

            Name (_CRS, ResourceTemplate ()  // _CRS: Current Resource Settings
            {
                IO (Decode16,
                    0x0060,             // Range Minimum
                    0x0060,             // Range Maximum
                    0x00,               // Alignment
                    0x01,               // Length
                    )
                IO (Decode16,
                    0x0064,             // Range Minimum
                    0x0064,             // Range Maximum
                    0x00,               // Alignment
                    0x01,               // Length
                    )
                IRQNoFlags ()
                    {1}
            })
            Name (_PRS, ResourceTemplate ()  // _PRS: Possible Resource Settings
            {
                StartDependentFn (0x00, 0x00)
                {
                    IO (Decode16,
                        0x0060,             // Range Minimum
                        0x0060,             // Range Maximum
                        0x00,               // Alignment
                        0x01,               // Length
                        )
                    IO (Decode16,
                        0x0064,             // Range Minimum
                        0x0064,             // Range Maximum
                        0x00,               // Alignment

Actually I had updated the bios to now version 403 (last one with working keyboard was 308)

and now funnily it's totally unresponsive again

> Actually I had updated the bios to now version 403 (last one with working
> keyboard was 308)

Do you mean with *partially* working keyboard? (Or did I miss something, that it worked fine?)

No you got that right. was partially working (initial bootup to some time after that)
under grub, it actually works fine too

Hey Paul,

Sorry that with not working again, seemed to be my fault.
Had run update and updated the kernel.
Just swapped back to the asus patch kernel and it works again.
Just logged in with the keyboard on the laptop, but it hangs after some time again and repeats letters when typing.

Will try the options given by Tamim and feedback again with demsg

Would it also be possible for you attach the dmesg or `dmesg | grep i8042` right after the keyboard dies after its been working for a while? We can probably use the i8042 logs to figure out how the communication is breaking down and try to patch that.

Tamim

Created attachment 304237 [details]
dmesg with dumbkbd

Hey Tamim,

Actually will not be possible, already tried, but also my external keyboard stops working somehow, after the internal trips.
I tried by now each i80442 option seprately and get the dmesg directly after boot.
All keys are now the right ones when i can type with the nternal one, before they were randomly worng.

but the repea error keeps persistent for me, after shortly orking, the internal one will let me write like this:
example word enter looks when typing like this:

eeeeeeeeeeeeennnnnnnnnnnnnnnnnnnnnttttttttttttttttttttttttttteeeeeeeeeeeeeeeeerrrrrrrrrr

Will continue with the diferent possible combinations and upload further.
I alos loaded openSUSE and will try ton run it asap.

Created attachment 304238 [details]
dmesg noaux option

Created attachment 304239 [details]
dmesg nomux option

The error above happens also after i deactivated repeat keys in accessibility menu

I tired opensuse yesterday evening, but the behavior is similar.
the keyboard works, for selecting installation options (i think it's under yast), but as soon as i come to the graphical installation menu, the keyboard isn't working anymore.

for suse also found a similar thread
https://lists.opensuse.org/archives/list/bugs@lists.opensuse.org/thread/O7JCC6R7ZQDTOOE6D7K42IFKAIN3J257/

Seems the combination of asus and ryzen is very bad for the linux world.

i installed windows again, just to try if something else broke eventually during all the installations etc.

but in windows all keys are working fine without any issues.

Ok guys,

Thanks Tamim and Paul for all the work here.
I will exchange the laptop with one of my workers, since he has almost the same model, but with intel chipset and geforce graphics.
I actually need the unit for work so might be better that way.

Let me know, if I can donate you both something?

I am experiencing what seems to be this same issue on an ASUS FA617XT: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1036004

Ubuntu 23.04LTS (kernel 6.2), Debian Bookworm RC2 (kernel 6.1 and experimental kernel 6.3), seem to behave as though the keyboard is entirely inoperable, though it seemed just extremely broken and sometimes inputs would occur and get stuck: generally though, it simply does not work, no input.

With older kernels, like Ubuntu 22.04LTS (kernel 5.19?) or Debian Stable (kernel 5.18?), the behavior is also bad: keys _might_ input and then get "stuck": essentially unusable.

The machine shipped with BIOS version FA617XT.303, and updating to the FA617XT.307 (mid April), does not resolve this issue.

I hope to try with builds of kernel 6.3.2, though I expect this won't resolve anything given the comments here.

Please let me know what I can do to help diagnose this issue: I don't believe ASUS nor AMD would be happy to simply have these machines non-functional with Linux for such a seemingly simple protocol/signalling issue.

As well, for me, kernels less than v6.3 suffer from kernel warning traces/taints from amdgpu; nomodeset option was the only way in Debian to achieve a booting Linux; Ubuntu seems to behave better and allow booting the system when it failed to init the GPUs (Ryzen 780M and Radeon RX 7700S), but Debian would simply reset the system during boot (or sometimes hang the system at a black screen, as though the reset failed): https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1035971

Hello, I am having the same issue and symptoms here on ASUS TUF FA507NU (AMD 7735HS/RTX4050): https://bugzilla.suse.com/show_bug.cgi?id=1211380#c0.

Kernel 6.3.1 and latest bios doesn't change anything for me on Tumbleweed. I will also be more than happy to do any tests to help fix this issue.

Nathan, Julien, please also contact ASUS about this issue. Maybe Seb can share a ticket number, you can reference.

I contacted them and link to this bug. I am waiting for their reply.

Created attachment 304271 [details]
ASUS Live Chat support

part 1/2 from ASUS Live Chat support about the issue -- expectedly unfruitful

> ...
> Kevin_L.: I do apologize as we do not assist with the Linux operating system.
> Me: I understand, but that's a disingenuous position: this issue is about
> supporting the hardware.
> Me: There's no one else you can refer me to for assistance?
> Kevin_L.: No I do apologize, we do not assist with the Linux operating
> system.
> ...
> Kevin_L.: Your case number for future reference is: N2305063195-0001 We do
> appreciate your understanding and we do value your business, thank you for
> being a part of the ASUS family.

Created attachment 304272 [details]
ASUS Live Chat support

part 2/2 from ASUS Live Chat support about the issue -- expectedly unfruitful

Created attachment 304286 [details]
Debian 6.3.0-0 amd64 i8042 option variations logs

Attached is a tarball of kernel logs and DSDT output, from Debian's experimental 6.3.0-0-amd64 kernel, with i8042.debug + i8042.dumbkbd/i8042.noaux/i8042.nomux permutations on an FA617XT w/ .307 BIOS version.  The kernel is booted with nomodeset because amdgpu also outputs IRQ warning traces when the GPUs are initialized; perhaps this is interesting.

I wish I had checked Windows 11 Home to see how the keyboard works; if it does and how it is attached (USB vs PS/2).

@Nathan
Yeah Asus feedback is indeed very nice. Had the same and I actually had gotten the account from my dealer (sells mainly Asus products).
I thought from his account it might have more weight, but I've got the same feedback.
I would try Win 11 just to check everything's works fine over there.
Actually reinstalled it for the guy I made the exchange with and all was good. Never had any issue with Keyboard there.
Does your's work in grub?
There is also a keyboard test tool in the bios actually.

Nathan, Julien, despite this probably being a Linux kernel shortcoming, I still recommend to return the device to show Asus, that they can’t sell these, when they do not verify that the Linux kernel works. (Especially when they lie about working GNU/Linux support.)

(In reply to seb.j from comment #85)
> Does your's work in grub?
> There is also a keyboard test tool in the bios actually.

Yes, the keyboard works well in GRUB, and presumably in the BIOS hardware test tool.

I wonder how it behaves on e.g. Windows 8, 10, vs the 11 it shipped with.

(In reply to Paul Menzel from comment #86)
> Nathan, Julien, despite this probably being a Linux kernel shortcoming, I
> still recommend to return the device to show Asus, that they can’t sell
> these, when they do not verify that the Linux kernel works. (Especially when
> they lie about working GNU/Linux support.)

Where did ASUS claim working GNU/Linux support?  I didn't see this, but instead simply that some models could be purchased without an operating system installed: note this was not an option for the FA617XT model in the US, just one configuration is/was available.

@Nathan,

I think Asus not really advertises this, but when I asked my dealer, he actually gave me the laptop as option (FA617NS) and told me it would be compatible.
I think that was just a failure by the dealer, while when I checked later, Asus also refused to support.
My dealer would have exchanged the unit, but since I bought a similar unit for one of my workers, that wasn't necesssary anymore (since he sticks to windows, i just exchanged the units and prepared the old one for him).

Long story short, my dealer advertised it as Linux compatible (i live in Thailand / not sure if that makes a difference)

On page 100 of the English manual edition it says:

> NOTE: Energy Star is NOT supported on FreeDOS and Linux-based operating
> systems

But otherwise it’s indeed not mentioned for the FA617XT. One reason more to not support such a vendor and return the device. (Sorry for going off-topic.)

If you still want to keep that device my last suggestion for this thread is to contact linux-input@vger.kernel.org and the maintainer (see `MAINTAINERS`) and make them aware of the problem. Please also carbon-copy Greg Kroah-Hartman, LKML, and the ACPI mailing list. I think, the INPUT subsystem does not use the Linux kernel Bugzilla bug tracker.

Hello, this kernel patch fixes the issue : https://lore.kernel.org/linux-acpi/20230518183920.93472-1-mario.limonciello@amd.com/

(In reply to drich from comment #90)
> Hello, this kernel patch fixes the issue :
> https://lore.kernel.org/linux-acpi/20230518183920.93472-1-mario.
> limonciello@amd.com/

I was able to test this patch against v6.3 in Debian.  It does not resolve the issue entirely for me; it does make the keyboard _slightly_ more functional: I can type my username at the prompt and perhaps 60-120 seconds later the characters will appear to register.  This seems like the same behavior w/ the 5.x kernels before the set of patches noted above were applied.

Happened to me also, didn't know if it was down to my setup or not.

I found a workaround :
- open root shell
- go to /sys/bus/platform/devices and search for ITE5570:00 in the AMDI00**/i2c-* folders (mine was in AMDI0010:01)
- run the following command : echo 'AMDI0010:01' > /sys/bus/platform/drivers/i2c_designware/unbind (change AMDI0010:01 for the one you found)

This should just fix the issue without side effect (still don't know what this ITE5570:00 is, its recognized as a generic-HID by kernel, and as a keyboard by libinput).

I created a oneshot systemd service to run it on boot as WantedBy multi-user.target

As this issue has a lot of comments, and is not only about the ** anymore, please always list your device.

1.  Nathan has a ASUS FA617XT.
2.  drich, what device do you have?

drich, thank you very much for your workaround. What distribution and Linux version did you test it with?

Oh yeah sorry, here are my system infos :

System:
  Host: laptop
  Kernel: 6.4.0-rc3-amd64+ (custom build)
  Cmdline: BOOT_IMAGE=/boot/vmlinuz-6.4.0-rc3-amd64+ root=UUID=9012f2a0-b3f6-49eb-ae42-a08feded24dc ro (default one)
  Distro: Debian GNU/Linux 12 (bookworm)
Machine:
  Type: Laptop
  System: ASUSTeK
  Product: ASUS TUF Gaming A16 FA617XS_TUF617XS v: 1.0
  Mobo: ASUSTeK FA617XS v: 1.0 serial: C5W33M100AY
  UEFI: American Megatrends LLC. v: FA617XS.303 date: 03/23/2023

(In reply to drich from comment #92)
> Happened to me also, didn't know if it was down to my setup or not.
> 
> I found a workaround :
> - open root shell
> - go to /sys/bus/platform/devices and search for ITE5570:00 in the
> AMDI00**/i2c-* folders (mine was in AMDI0010:01)
> - run the following command : echo 'AMDI0010:01' >
> /sys/bus/platform/drivers/i2c_designware/unbind (change AMDI0010:01 for the
> one you found)
> 
> This should just fix the issue without side effect (still don't know what
> this ITE5570:00 is, its recognized as a generic-HID by kernel, and as a
> keyboard by libinput).

This, along with the patch you shared, makes the keyboard work on my ASUS FA617XT.  The device is the same: AMDI0010:01.

Thanks.  How did you uncover this?

> How did you uncover this?

I had another problem : the keyboard was perfectly working with the patch but the touchpad was not detected at all on my setup, it appeared that I just had the I2C-HID modules disabled in my kernel config.

Once enabled, the keyboard started behaving the same way described in this thread, so at this point I knew it was caused by something on the I2C stack. I then played around with all the I2C peripherals and settings to finally find that AMDI0010:01 / ITE5570:00 was faulty.

FYI I've posted a v3 of the patch to revert the heuristic.  I don't expect this to help the issue with the i2c-hid version of the keyboard that Nathan and Drich had, but it should help PS/2 keyboards.

https://lore.kernel.org/linux-acpi/20230601221151.670-1-mario.limonciello@amd.com/T/#u

I have an Asus Fa617XT as well and experiencing this same issue and found this that might be helpful regarding this issue: https://lore.kernel.org/linux-input/20230530154058.17594-1-friedrich.vock@gmx.de/T/

*** Bug 217493 has been marked as a duplicate of this bug. ***

(In reply to Mario Limonciello (AMD) from comment #97)
> FYI I've posted a v3 of the patch to revert the heuristic.  I don't expect
> this to help the issue with the i2c-hid version of the keyboard that Nathan
> and Drich had, but it should help PS/2 keyboards.
> 
> https://lore.kernel.org/linux-acpi/20230601221151.670-1-mario.
> limonciello@amd.com/T/#u

v3, w/ the addition to disable the "GPIO 7" / ITE5570 device, continues to work for me on 6.3.5.

Hello, I have an Asus FA507NV laptop. I installed Linux Mint 21.1 with kernel version 5.15.0-56, and the keyboard works fine. However, after updating the kernel to version 5.15.0-73, it stops working.

Update:

My ASUS FA617XT is completely dead, no signs of life: will not power on, does not seem to charge from AC adapter or USB Type-C port; dis/reconnecting the battery has no effect, nor removing the RAM DIMMs or NVMe cards (SSD, WiFi).

I was running v6.4-rc4 w/ v3 patch, and with the ITE 5570 device unbound.  After using the machine (GPU intensive task) and letting it idle for many hours, it suddenly powered off and now appears lifeless, a semi-expensive fancy-looking brick.

I'll be contacting ASUS for warranty support.  I suspect this is simply a hardware issue with my specific device, but wanted to share in case others are running v6.4 and/or the above patches.

@Nathan

eventually check for thermal management issues, i actually had my laptop ran very hot in the beginning and found that the thermal management wasn't working correctly, but ofr me it was fixed with kernel 6.3.
sorrry didn't keep track of the specific error.

I think it was related to the amd-gpu error, i had at startup, when now remembering.
eventually your gc burned itself?

Using 6.4.y with the patches too 👀 , will avoid GPU intensive tasks for now.

Just installed a second SSD with Windows 11 on it, keyboard works perfectly well on both installer and the installed system.

The keyboard is detected on the same PS/2 interface as Linux, and ITE5570 as a generic "I2C HID Device" (don't know how to gather more precise informations)

FWIW, I don't think the issue is GPU related, though it could be.  I may pop out the SSD and try to check the logs; I think the amdgpu_irq_put kernel taint warnings were still occurring but unsure: OpenCL via ROCm was working great, just ran a task for maybe 15m and then left the laptop on afterward.  The fans would throttle up and down with the load.

Thanks for checking in Windows; I was tempted to install a second SSD as well.  I wiped the SSD that came with it; I wonder if it's possible to recover my license or how that works.

I'll be contacting ASUS on Monday about this all.

> The keyboard is detected on the same PS/2 interface as Linux, and ITE5570 as
> a generic "I2C HID Device" (don't know how to gather more precise
> informations)

Could you 
> I think the amdgpu_irq_put kernel taint warnings were still occurring

The last of these for Phoenix should be fixed in 6.4-rc5.

> I wiped the SSD that came with it; I wonder if it's possible to recover my
> license or how that works.

It should be stored in BIOS and found automatically by the Windows installer.

> The keyboard is detected on the same PS/2 interface as Linux, and ITE5570 as
> a generic "I2C HID Device" (don't know how to gather more precise
> informations)

On 
https://lore.kernel.org/linux-input/20230530154058.17594-1-friedrich.vock@gmx.de/T/ I was looking for a few things.  Most notably:

1) The output from /sys/kernel/debug/gpio
2) Sleuthing to figure out which interrupt is causing the storm in /proc/interrupts, the master interrupt or the one tagged to that GPIO
3) Use R/W everything in Windows to check the GPIO register configuration against Linux.  For that individual register the address will be in /sys/kernel/debug/gpio in Linux and you can use the MMIO function in R/W everything to access it in Windows.

> Could you 

My apologies, I forgot to delete this block, I completed my thought at the end of the comment.

I'm having the same problem with marslogout@gmail.com with the same model. There's a post on opensuse.org forum (https://forums.opensuse.org/t/keyboard-not-working-on-laptop-amd-asus-tuf/165741/14) which seems to make it work for them by commenting out some lines in drivers/acpi/resource.c. Maybe this can help?

> I'm having the same problem with marslogout@gmail.com with the same model.
> There's a post on opensuse.org forum
> (https://forums.opensuse.org/t/keyboard-not-working-on-laptop-amd-asus-tuf/165741/14)
> which seems to make it work for them by commenting out some lines in
> drivers/acpi/resource.c. Maybe this can help?

This patch which is under testing will fix it.

https://lore.kernel.org/linux-acpi/20230601221151.670-1-mario.limonciello@amd.com/T/#u

Not too familiar with patching up the kernel myself. Will try to test it over the week. But will this be expected to be in 6.4?

> Not too familiar with patching up the kernel myself. Will try to test it over
> the week. But will this be expected to be in 6.4?

It depends on how much testing it gets; but I think unlikely.

(In reply to Mario Limonciello (AMD) from comment #112)
> > Not too familiar with patching up the kernel myself. Will try to test it
> over
> > the week. But will this be expected to be in 6.4?
> 
> It depends on how much testing it gets; but I think unlikely.

So, you mean to say that the more people test this patch, the higher the likelihood that it will be included in the upcoming kernel releases?

Does this patch work for any kernel version or only for specific ones? Can you provide a link to an article that describes the patch installation process step by step? It would be quite difficult for an inexperienced user to do it without proper instructions.

> So, you mean to say that the more people test this patch, the higher the
> likelihood that it will be included in the upcoming kernel releases?

I explicitly said in the patch it needs testing from a majority of people before it should go in.  It's risky otherwise.

> Does this patch work for any kernel version or only for specific ones? Can
> you provide a link to an article that describes the patch installation
> process step by step? It would be quite difficult for an inexperienced user
> to do it without proper instructions.

Contextually it might change from one version to another depending upon what other commits are there already.

In terms of building a kernel, you should reference your distro for the best directions.  For example Fedora has this:
https://docs.fedoraproject.org/en-US/quick-docs/kernel/build-custom-kernel/

I've built with versions of the patch against 6.3, 6.4, and 6.1 IIUC.

On Debian, there is a utility to build a kernel with a set of patches with a single command; this is the guide:

https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#id-1.6.6.4

I had to use `--fuzz 3` due to some slight differences, but it applied cleanly.  The `b4` command (and an existing Git repository) will allow to create a .patch if copy/paste from the mail doesn't work.

Is this line in drivers/acpi/resource.c really needed? Seems to be present in the stable releases but already removed in linux-next.

#ifdef CONFIG_X86
	/*
	 * IRQ override isn't needed on modern AMD Zen systems and
	 * this override breaks active low IRQs on AMD Ryzen 6000 and
	 * newer systems. Skip it.
	 */
	if (boot_cpu_has(X86_FEATURE_ZEN))
		return false;
#endif

(In reply to Mario Limonciello (AMD) from comment #114)
> > So, you mean to say that the more people test this patch, the higher the
> > likelihood that it will be included in the upcoming kernel releases?
> 
> I explicitly said in the patch it needs testing from a majority of people
> before it should go in.  It's risky otherwise.

Hello,
just wanted to confirm the latest version of your patch working with the Asus FA617XS variant, together with the I2C unbind fix on the Ubuntu 23.04&kernel 6.4-rc6.
Touchpad became unresponsive after moving to this kernel, but the unbind/bind of AMDI0010:00 did the trick.

What I have noticed, when I dropped out to the initramfs shell with the patched kernel, the keyboard was working perfectly even without the I2C unbind. But after I got to the graphics, it had effectively stopped working and needed to be unbound. Not sure if it is relevant, just sharing an observation.
Thank you and everyone else for getting my laptop running!

(In reply to Nik_P from comment #117)
> (In reply to Mario Limonciello (AMD) from comment #114)
> > > So, you mean to say that the more people test this patch, the higher the
> > > likelihood that it will be included in the upcoming kernel releases?
> > 
> > I explicitly said in the patch it needs testing from a majority of people
> > before it should go in.  It's risky otherwise.
> 
> Hello,
> just wanted to confirm the latest version of your patch working with the
> Asus FA617XS variant, together with the I2C unbind fix on the Ubuntu
> 23.04&kernel 6.4-rc6.
> Touchpad became unresponsive after moving to this kernel, but the
> unbind/bind of AMDI0010:00 did the trick.

Thanks!  It's queued up for 6.5.

> 
> What I have noticed, when I dropped out to the initramfs shell with the
> patched kernel, the keyboard was working perfectly even without the I2C
> unbind. But after I got to the graphics, it had effectively stopped working
> and needed to be unbound. Not sure if it is relevant, just sharing an
> observation.
> Thank you and everyone else for getting my laptop running!

Do you have i2c-hid in your initramfs?  I would think it's not tied to graphics specifically, but rather that i2c-hid is in your rootfs instead of initramfs so issue didn't trigger until i2c-hid loaded.

If possible; do you or anyone else have a dual boot with Windows?  I would really like if you can compare the configuration for the problematic GPIO in Windows (using R/W everything) and Linux (using /sys/kernel/debug/gpio).

(In reply to Nik_P from comment #117)
> (In reply to Mario Limonciello (AMD) from comment #114)
> > > So, you mean to say that the more people test this patch, the higher the
> > > likelihood that it will be included in the upcoming kernel releases?
> > 
> > I explicitly said in the patch it needs testing from a majority of people
> > before it should go in.  It's risky otherwise.
> 
> Hello,
> just wanted to confirm the latest version of your patch working with the
> Asus FA617XS variant, together with the I2C unbind fix on the Ubuntu
> 23.04&kernel 6.4-rc6.
> Touchpad became unresponsive after moving to this kernel, but the
> unbind/bind of AMDI0010:00 did the trick.
> 
> What I have noticed, when I dropped out to the initramfs shell with the
> patched kernel, the keyboard was working perfectly even without the I2C
> unbind. But after I got to the graphics, it had effectively stopped working
> and needed to be unbound. Not sure if it is relevant, just sharing an
> observation.
> Thank you and everyone else for getting my laptop running!

Hello, I also encountered a similar issue when installing the Nvidia proprietary driver with open-source code. There were no problems when installing the driver with closed-source code, so you can check that. Keep in mind that simply reinstalling the driver doesn't help. I had to reinstall the system to verify this.

I confirm the issue on FA507NU laptop.
The funny thing is that after installation of current Linux Mint 21.1 with 5.15 Kernel, everything worked fine. But after switching to Kernel 5.19, the keyboard stopped working. 

I have two pieces of the same model - it is the same for both of them.

The discussion is quite confusing for me. What is the current status? What mainline kernel version will have the fix?

The patch will be landing in kernel 6.5-rc1.  If no other problems are reported we'll intend to bring it back to stable later on as well.

> The discussion is quite confusing for me. What is the current status? What
> mainline kernel version will have the fix?

There is a separate issue going on where an I2C device is causing an interrupt storm which makes the keyboard seem like it's not working.  Still looking for debugging information on that to come up with a proper solution.

Thanks, Mario.
I thereby confirm that your patch applied on latest v6.4 tag works on FA507NU.

Apart from a few ACPI bugs in dmesg (I have never ever seen a system without some) I had no problems so far. Is there anything I should focus on during my testing?

> Is there anything I should focus on during my testing?

If you are affected by the interrupt storm, I would like to see a comparison of registers from R/W everything.

Created attachment 304477 [details]
/sys/kernel/debug/gpio

Here is the content of my /sys/kernel/debug/gpio "file", but I don't understand how R/W Everything works. How should I set it up to get the GPIO output from Windows side ?

For using R/W everything in Windows you will use the "MMIO" function.  
1. Press the MMIO button and then hit the "+" button.
2. Add a mapping for the base address (FED81500) and save it.
3. Add line items for all the relevant GPIO registers and their offsets from that base address.

For example if you wanted to look at pin 7, it should be 1C.
The master register is FC (See https://github.com/torvalds/linux/blob/master/drivers/pinctrl/pinctrl-amd.h#L19).

At a minimum we want to match the values in R/W everything against those in /sys/kernel/debug/gpio for:
1. the GPIO that causes the storm
2. the power button interrupt (offset 0)
3. the master controller register (offset (FC)

The names aren't exactly the same platform but as it's the most open document I know of and the offsets don't change you can reference https://www.amd.com/system/files/TechDocs/50742_15h_Models_60h-6Fh_BKDG.pdf to try to find the offsets if you have a hard time for your problematic GPIO.

Created attachment 304483 [details]
FA617XT.309 gpio, interrupts, RWEverything-mmio

Attached are `/proc/interrupts`, `/sys/kernel/debug/gpio`, and the output from RW Everything utility, from my repaired ASUS FA617XT, which "conveniently" came back with Windows 11; the captures are from Ubuntu 23.04 install/live CD.

I'm not sure which interrupt is the noisy one, so I captured as much as I could: please let me know if this captures what's necessary, or what else I should look at.  Without your (Mario) notes about the GPIO base address, I wouldn't have been able to get this.  I captured banks 0, 1, and 2, though `.../gpio` lists a third bank not listed in "BKDG for AMD Family 15h Models 60h-6Fh Processors."

----

For those interested, the repair sheet said they replaced the MAIN BOARD ("no power on (LED indicators did not light up)") and the KEYBOARD MODULE ("certain key pressed and triggered multiple response").  I don't think I took good enough notes to tell if anything else was replaced or if any of this is original (beside the S/N / sticker).

Also, I was able to capture the kernel log from when the system turned into a brick, and there's some interesting output from amdgpu; as well it almost seems like systemd restarted or the system tried to reinitialize without power cycling somehow.  I'll share them later if anyone is interested, or can help assure me it won't happen again due to some misconfiguration or use ...

Created attachment 304485 [details]
PDF of GPIO register configuration; Win 11 vs Linux

I did some analysis on the captured values (FA617XT.309); see the last page for comparison: non-zero values are differences (simple subtraction: red lettering indicating the Win 11 value is "larger" -- I'm aware decimal comparison isn't ideal ...).

Hopefully this helps; there are a number of registers that are configured differently.

Hi,

FA617NS here, and some registers are also configured differently here. I *think* the GPIO pin that causes the storm might be pin 7, so I captured that pin, pin 0 and the master controller register as well as some random others in R-W everything and on Linux. It looks something like this:

pin 0: 805F800 on Windows, 815F8E3 on Linux. Windows clears bit 20 and the whole first byte, Linux sets bit 20 and some of the first byte.

pin 1: 150000 on both Windows and Linux

pin 2: 15EB00 on both Windows and Linux

pin 3: 150000 on both Windows and Linux

pin 4: 41940 on Windows, 247C00 on Linux. Different bits are bits 21, 14, 13, 8 and 6.
Windows clears bits 21, 14 and 13 while Linux sets them, and Linux clears bits 8 and 6 while Windows sets them.

pin 5: 240000 on both Windows and Linux

pin 6: 240000 on both Windows and Linux

pin 7: 151B00 on Windows, 10040B00 on Linux. Different bits are bits 28, 20, 16 and 12.
Windows clears bit 28 while Linux sets it, and Linux clears bits 20, 16 and 12 while Windows sets them. Bit 28 is interesting, according to the docs, if it is cleared, it doesn't generate interrupts. Sounds like the storm might go away if that bit is cleared?

Master control register: FF00000000 on both Windows and Linux.

Created attachment 304487 [details]
potential patch (v1)

It looks like pin 7 has the same results for both of you guys; let's focus on that.

> pin 0: 805F800 on Windows, 815F8E3 on Linux. Windows clears bit 20 and the
> whole first byte, Linux sets bit 20 and some of the first byte.

OK I expect pin 0 is closer to the same after the series going to 6.5-rc1 hopefully.  This shouldn't affect this issue.

> pin 7: 151B00 on Windows, 10040B00 on Linux. Different bits are bits 28, 20,
> 16 and 12.
> Windows clears bit 28 while Linux sets it, and Linux clears bits 20, 16 and
> 12 while Windows sets them. Bit 28 is interesting, according to the docs, if
> it is cleared, it doesn't generate interrupts. Sounds like the storm might go
> away if that bit is cleared?

bit 28: Linux does write the GPIO again to clear it.
https://github.com/torvalds/linux/blob/v6.4/drivers/pinctrl/pinctrl-amd.c#L668C18-L668C18

bit 20: This is most concerning to me.  Windows configured the pin as pull up, but Linux didn't.

bit 16: this is read only; just a status bit based on current status.

bit 12: this controls the interrupt delivery.  It appears that interrupt is enabled on Windows but not on Linux.

So I think we should be focusing on bit 20 to figure out why that got programmed differently.

I don't see in the spec anything about bit 19 being used for pull up select.  I wonder if we've been programming bit 19/20 incorrectly as a result.

Please try this patch.

This patch doesn't seem to change anything. The issue is still there, and the GPIO register value seems the same as well.

Can you please share an acpidump from your system?

Actually assuming it's the same as https://bugzilla.kernel.org/show_bug.cgi?id=217493 then it should have been configured as a PullUp.

                    GpioInt (Level, ActiveLow, Exclusive, PullUp, 0x0000,
                        "\\_SB.GPIO", 0x00, ResourceConsumer, ,
                        )
                        {   // Pin list
                            0x0007
                        }

I'll keep looking around for where the bug is.

I wonder if this is a situation that the interrupt for the GPIO is masked because no driver binds to the ACPI device.  
To rule out if we still have a problem in pinctrl-amd code, can you please share 
1. the output /sys/kernel/debug/gpio
and
2. of this with my bit 19/20 patch applied:

diff --git a/drivers/pinctrl/pinctrl-amd.c b/drivers/pinctrl/pinctrl-amd.c
index 9f7fabaffef59..b808af7752dcf 100644
--- a/drivers/pinctrl/pinctrl-amd.c
+++ b/drivers/pinctrl/pinctrl-amd.c
@@ -783,6 +783,8 @@ static int amd_pinconf_set(struct pinctrl_dev *pctldev, unsigned int pin,
                arg = pinconf_to_config_argument(configs[i]);
                pin_reg = readl(gpio_dev->base + pin*4);

+               pr_info("pinconf_set: pin %d, param %d, arg %d\n",
+                       pin, param, arg);
                switch (param) {
                case PIN_CONFIG_INPUT_DEBOUNCE:
                        pin_reg &= ~DB_TMR_OUT_MASK;

Created attachment 304508 [details]
potential patch (v2)

I've found the reason that potential patch v1 wasn't calling that function.  I've adjusted some of the calling structure.

Can you guys please test "potential patch (v2)"?

Note: if this works on 6.4/6.3.y, I'll be splitting it into multiple patches and rebase it on pinctrl.git.

(In reply to Mario Limonciello (AMD) from comment #135)
> Can you guys please test "potential patch (v2)"?

Mario, with this patch applied atop the first (https://lore.kernel.org/linux-acpi/20230601221151.670-1-mario.limonciello@amd.com/T/#u), my keyboard is now working without having to perform any extra dancing.  Linux v6.4-rc4 + patches (ASUS FA617XT.309).

What I find interesting is that these two interrupts seem to share the same GPIO or such; with this patch, #61 (amd_gpio 7) has many fewer entries than without (in fact it seems stable at 2 in my limited testing):

            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7       CPU8       CPU9       CPU10      CPU11      CPU12      CPU13      CPU14      CPU15······
   7:          0          0          0          0          0          0       8031          0          0          0          0          0          0          0          0          0  IR-IO-APIC    7-fasteoi   pinctrl_amd
  61:          0          0          0          0          0          0          2          0          0          0          0          0          0          0          0          0  amd_gpio    7  ITE5570:00

versus before:

            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7       CPU8       CPU9       CPU10      CPU11      CPU12      CPU13      CPU14      CPU15······
   7:          0          0          0          0          0          0     262745          0          0          0          0          0          0          0          0          0  IR-IO-APIC    7-fasteoi   pinctrl_amd
  44:          0          0          0          0          0          0     262413          0          0          0          0          0          0          0          0          0  amd_gpio    7

Can I please see updated /sys/kernel/debug/gpio output and an acpidump from your specific system?  I want to make sure this still lines up with what the firmware had intended.

Created attachment 304511 [details]
ACPI dump from FA617NS

Hi,

I can confirm the v2 patch also fixes the issue for me.

I attached an ACPI dump from my system. I also looked at the /sys/kernel/debug/gpio values again. Compared to last time I checked, some values changed, but most of them were due to other changes unrelated to this patch. This patch only affects the value of pin 7, changing it from the erroneous value from before to 151B00 (i.e. the same as Windows).

Thanks a lot for doing the proper fix for this!

Created attachment 304512 [details]
FA617XT.309 gpio, acpidump -- w/ both patch sets (on v6.4-rc4)

Mario, attached is the acpidump and /sys/kernel/debug/gpio from my v6.4-rc4 kernel on FA617XT.309 w/ the two patches.

Thanks guys!

I've split up the series and rebased on pinctrl.git and sent it up for review:
https://lore.kernel.org/linux-gpio/20230630194716.6497-1-mario.limonciello@amd.com/T/#t

Did this patch make it to 6.4.1? Sorry not too familiar with trying out patches. I tried it yesterday with 6.4 on Ubuntu 23.04 using mainline kernel but keyboard is still not working. I plan to try 6.4.1. Any logs that I can post to help?

It's in Linus' tree, and it has been submitted for a backport request to 6.4.2.  You can pick it now on top of 6.4.1 if you would like.

But please keep in mind that only fixes the keyboard, if you want to fix the interrupt storm you'll need the other series or other workaround.

Got it. Thanks again for all the hard work. Will try to report what I find out similar to the ones posted here.

Curious ... what's it take to get the vendors (ASUS in this case) to ship a correct DSDT with their products?

(In reply to Mario Limonciello (AMD) from comment #143)
> It's in Linus' tree, and it has been submitted for a backport request to
> 6.4.2.  You can pick it now on top of 6.4.1 if you would like.

I have an ASUS TUF Gaming A16 FA617NS. If I build the kernel from the master branch of linux-torvalds. Should that include the fix?

I tried booting it but the keyboard still didn't work.

The fix for the keyboard is landed in Linus' tree, but the interrupt storm solution isn't.  It's possible that you're hitting the interrupt storm making it seem the keyboard isn't working.

If you check out the fixes branch of https://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-pinctrl.git/ it should get you both solutions.

Thanks! The keyboard now works!

If possible, can you reply to this thread https://lore.kernel.org/linux-gpio/994042dd-9532-0dfb-8f86-98c897db75fd@amd.com/

with a tag like this:

Tested-by: Jan Visser <public_email_you_want_to_use>

All of these fixes are landed into 6.5-rc2, closing this issue.

Thank you so much Mario, all.  I'll give it a build soon; glad this is resolved.

Still curious what it takes to get ASUS et al to play ball, but ... so it goes.

Thank you Mario and all who contributed! Might finally get my laptop to work with this. Just curious, will this be backported to 6.4.3?

Thank you again.

> Just curious, will this be backported to 6.4.3?

Sure, I don't see a problem with doing this.  I just tested it and found a new error message from this series so I sent this up:

https://lore.kernel.org/linux-gpio/20230717201652.17168-1-mario.limonciello@amd.com/T/#u

After testing 6.1.y and 6.4.y I've submitted the following to request a backport:
https://lore.kernel.org/stable/10fef988-8460-f96a-2d3d-cc8c6a37f3eb@amd.com/T/#u

That's great! If you need more testing just let me know!

Any more testing is definitely welcome.  You can backport the patches to 6.1.y or 6.4.y as I described there.

(In reply to Mario Limonciello (AMD) from comment #156)
> Any more testing is definitely welcome.  You can backport the patches to
> 6.1.y or 6.4.y as I described there.

I have tryed and when i do make -j 16 it gives me **make [Makefile:2032: .] Error2 and I can´t find how to solve it. Also I don't know what's the beggining and the final of the patch. 
thank you

It's landed in stable 6.1 and 6.4 kernels now, if you upgrade to latest stable you'll have it.  It's also on it's way to 5.15.y and 5.10.y as well.

Created attachment 304693 [details]
dmesg - Asus A15 2023 - generic 6.4.6

I am still running 6.4.2+ with your first patch:
https://lore.kernel.org/linux-acpi/20230601221151.670-1-mario.limonciello@amd.com/T/#u

It does not seem to be in current stable yet. However, I am not able to start 6.4.6 anyway as it just freezes during the boot.

Attached dmesg shows the freeze for approx 3 minutes until I hard shut the PC (and ACPI power button press was detected)

During the boot the cursor stopped blinking, I could not switch to TTY and it did not even react to CTRL+ALT+SysRq commands...

Another difference to working 6.4.2 is that "amd_gpio AMDI0030:00: Invalid config param 0014" now show up during the boot.

Should I maybe open up another bug here?

> It does not seem to be in current stable yet.

Everything for this issue should be in 6.4.y now.

> Another difference to working 6.4.2 is that "amd_gpio AMDI0030:00: Invalid
> config param 0014" now show up during the boot.

https://bugzilla.kernel.org/show_bug.cgi?id=217698

> Attached dmesg shows the freeze for approx 3 minutes until I hard shut the PC
> (and ACPI power button press was detected)
> Should I maybe open up another bug here?

Please bisect between 6.4.2 and 6.4.6 and open a new bug with the result.  If it turns out to be caused by anything in drm or amdgpu please open the bug at https://gitlab.freedesktop.org/drm/amd

Thanks. I will try to find the commit that causes the issue.

However, your IRQ override patch is definitely not in stable 6.4.y right now:

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/acpi/resource.c?h=linux-6.4.y

Ohhh..  it's in the queue but the queue didn't empty yet is why.

https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git/tree/queue-6.4/acpi-resource-remove-zen-specific-match-and-quirks.patch?id=e6df090a412868dfb3cb5255e553f54167b61c97

(In reply to Marek Šanta from comment #159)
> Another difference to working 6.4.2 is that "amd_gpio AMDI0030:00: Invalid
> config param 0014" now show up during the boot.
> 
> Should I maybe open up another bug here?

I started noticing this as well about the same time (haven't tried 6.4.6 yet; Debian has only 6.4.4 and 6.4.5 has the fixes here) but unsure what it means.  If you do open a bug report for this, please link it.

BTW It's possible it's not kernel but instead GPU F/W. You can try to revert https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/commit/?id=cd524606ce5b43745c16554de8a5a5476de0f557 to see if that helps.

I did it with kernel 6.4 and the lasts 3 sentences are:
  LD [M]  drivers/comedi/drivers/ni_routing.o
  LD [M]  drivers/infiniband/hw/hfi1/hfi1.o
make: *** [Makefile:2026: .] Error 2

Now its not 2032, but its another line, i don't know what to do... i'm a bit exhausted and frustrated...

Can confirm that it's NOT working on 6.4.6 but working on 6.5rc3 (Keyboard's working). Probably worth noting as well that 6.5rc3 still breaks nvidia drivers 4xx - 5xx on Ubuntu 23.04.

I tryed all kind of things and the best one is downloading this https://mega.nz/file/6qYkiIZQ#Y8w79Ilh0n61B-A_8O7v5fg6lMKYLeDXg1LCOXg8kZU and installing all three .dev 
and before reboot, putting this command udo apt-get install build-essential libncurses-dev bison flex libssl-dev libelf-dev

the only thing i have now is that i need to know how to enter whithout recovering mode, anyone knows?

Unfortunately the dropping of:

```
#ifdef CONFIG_X86
	/*
	 * IRQ override isn't needed on modern AMD Zen systems and
	 * this override breaks active low IRQs on AMD Ryzen 6000 and
	 * newer systems. Skip it.
	 */
	if (boot_cpu_has(X86_FEATURE_ZEN))
		return false;
#endif
```

As done in commit a9c4a912b7dc ("ACPI: resource: Remove "Zen" specific match and
quirks") is causing keyboard problems for quite a lot of AMD based laptop users using other models, see e.g. :

https://bugzilla.redhat.com/show_bug.cgi?id=2228891
https://bugzilla.redhat.com/show_bug.cgi?id=2229165
https://bugzilla.redhat.com/show_bug.cgi?id=2229317
https://bugzilla.kernel.org/show_bug.cgi?id=217718
https://bugzilla.kernel.org/show_bug.cgi?id=217726
https://bugzilla.kernel.org/show_bug.cgi?id=217731

So I have submitted a revert upstream:
https://lore.kernel.org/linux-acpi/20230808103335.95339-1-hdegoede@redhat.com/

I did add an extra patch on top:
https://lore.kernel.org/linux-acpi/20230808103335.95339-4-hdegoede@redhat.com/

which should keep the behavior of a9c4a912b7dc for any laptop with a line like this in dmesg:

[    0.021867] ACPI: INT_SRC_OVR (bus 0 bus_irq 1 global_irq 1 low edge)

Note the "bus_irq 1" is important here. There will also be other INT_SRC_OVR messages with a different bus_irq!

If possible please give this patch-series a try:
https://lore.kernel.org/linux-acpi/20230808103335.95339-1-hdegoede@redhat.com/

With this series there should still be a line like:

[    0.410670] ACPI: IRQ 1 override to edge, low(!)

in dmesg for laptops which need the override (based on the INT_SRC_OVR bus_irq 1),
so this patch-series should not cause a regression on the laptop from the original reporter of this bug.

I confirm that after applying your set of three patches on top of current 6.4.9 stable, the keyboard on ASUS FA507NU still works well! :)

the line

ACPI: IRQ 1 override to edge, low(!)

is present in my dmesg

I replicate my other post on a very similar thread just to point out what follows: I solved the problem pretty easily after racking my brains for several weeks. My B150 series Asus ExpertBook keyboard, otherwise  perfectly working, stopped responding once Linux had booted. I took the following steps:

I added this repository:

sudo add-apt-repository ppa:damentz/liquorix && sudo apt-get update

I downloaded the kernel 

sudo apt-get install linux-image-liquorix-amd64 linux-headers-liquorix-amd64

No need to add patches or anything, keyboard was right away alive and kicking!