Initially reported to systemd, but they told me this is a uinput/kernel bug and asked me to open a bug report here. As a reference: https://github.com/systemd/systemd/issues/15784 I'm running ckb-next-daemon which creates input devices for corsair peripherials. On each udevadm trigger I get errors with "Cannot allocate memory", which seem to come from kobject_uevent.c. Though, my knowledge does not permit me why this error occurs, so I went from ckb-next to systemd/udev, and ended up here where I hope somebody can take a look on why this error occurs. One of the ckb-next devs was kind enough to make a trace for me: [40677.318139] ------------[ cut here ]------------ [40677.318141] add_uevent_var: buffer size too small [40677.318183] WARNING: CPU: 2 PID: 25639 at lib/kobject_uevent.c:670 add_uevent_var+0xfa/0x110 [40677.318183] Modules linked in: pci_stub vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) binfmt_misc kvm_amd kvm mxm_wmi irqbypass rc_pixelview snd_emu10k1_synth snd_emux_synth crct10dif_pclmul snd_seq_midi_emul crc32_pclmul snd_seq_virmidi ghash_clmulni_intel tuner_simple tuner_types snd_emu10k1 snd_hda_codec_hdmi joydev tuner input_leds tda7432 snd_hda_intel aesni_intel tvaudio snd_util_mem snd_hda_codec snd_ac97_codec bttv ac97_bus snd_hda_core aes_x86_64 glue_helper crypto_simd snd_hwdep tveeprom snd_pcm_oss videobuf_dma_sg snd_mixer_oss cryptd videobuf_core tea575x snd_pcm amdgpu rc_core v4l2_common videodev fam15h_power mc emu10k1_gp gameport snd_seq_midi snd_seq_midi_event sp5100_tco snd_rawmidi gpu_sched snd_seq i2c_algo_bit ttm snd_seq_device drm_kms_helper snd_timer syscopyarea sysfillrect sysimgblt fb_sys_fops drm snd parport_serial soundcore wmi vhba(OE) parport_pc ppdev lp parport autofs4 hid_generic usbhid hid uas usb_storage i2c_piix4 tg3 firewire_ohci ahci [40677.318216] firewire_core pata_acpi libahci floppy [40677.318220] CPU: 2 PID: 25639 Comm: udevadm Tainted: G OE 5.3.8-uwuvyouwofty #1 [40677.318221] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./990FX Extreme4, BIOS P2.70 06/05/2014 [40677.318223] RIP: 0010:add_uevent_var+0xfa/0x110 [40677.318224] Code: 48 83 c4 50 5b 41 5c 5d c3 48 c7 c7 c8 fe 44 8d e8 a4 ad 69 ff 0f 0b b8 f4 ff ff ff eb d2 48 c7 c7 f0 fe 44 8d e8 8f ad 69 ff <0f> 0b b8 f4 ff ff ff eb bd e8 88 aa 69 ff 0f 1f 84 00 00 00 00 00 [40677.318225] RSP: 0018:ffffa68042cf7d40 EFLAGS: 00010286 [40677.318227] RAX: 0000000000000000 RBX: ffff9573ee690000 RCX: 0000000000000006 [40677.318227] RDX: 0000000000000007 RSI: 0000000000000096 RDI: ffff95751ea974d0 [40677.318228] RBP: ffffa68042cf7da0 R08: 0000000000000542 R09: 0000000000000034 [40677.318229] R10: 0000000000000000 R11: ffffa68042cf7be8 R12: 0000000000000005 [40677.318230] R13: ffff95751ac75300 R14: ffff9573e832f218 R15: 0000000000000000 [40677.318231] FS: 00007f27fc15b880(0000) GS:ffff95751ea80000(0000) knlGS:0000000000000000 [40677.318232] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [40677.318233] CR2: 000055c381d8bcf0 CR3: 000000049a2c8000 CR4: 00000000000406e0 [40677.318234] Call Trace: [40677.318239] ? dev_uevent+0xba/0x2e0 [40677.318241] kobject_uevent_env+0x360/0x790 [40677.318244] ? do_filp_open+0xa7/0x100 [40677.318245] kobject_synth_uevent+0x2bb/0x32b [40677.318247] uevent_store+0x1c/0x30 [40677.318249] ? _cond_resched+0x15/0x30 [40677.318251] kernfs_fop_write+0x108/0x190 [40677.318253] vfs_write+0xa5/0x1a0 [40677.318255] ksys_write+0x59/0xd0 [40677.318257] do_syscall_64+0x55/0x130 [40677.318260] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [40677.318262] RIP: 0033:0x7f27fc039317 [40677.318263] Code: 64 89 02 48 c7 c0 ff ff ff ff eb bb 0f 1f 80 00 00 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24 [40677.318264] RSP: 002b:00007fff329c4948 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [40677.318266] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f27fc039317 [40677.318266] RDX: 0000000000000007 RSI: 00007fff329c4a00 RDI: 0000000000000003 [40677.318267] RBP: 00007fff329c4a00 R08: 0000000000000007 R09: 7269762f73656369 [40677.318268] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000007 [40677.318269] R13: 000055c381dc97c0 R14: 0000000000000007 R15: 00007f27fc1138a0 [40677.318270] ---[ end trace 0412a24c9ab61437 ]--- [40677.318272] synth uevent: /devices/virtual/input/input71: failed to send uevent [40677.318273] input input71: uevent: failed to send synthetic uevent I hope that's enough detail for you to make sense of it, if you require further detail - I'll do my best to help. Thanks in advance for reading / analyzing, Georg
Hi I'm facing a similar issue on Debian/unstable: $ sudo udevadm trigger --action=change Failed to write 'change' to '/sys/devices/virtual/input/input30/uevent': Cannot allocate memory Here's the involved udev rule from lcdproc packages: $ cat /lib/udev/rules.d/60-lcdproc.rules KERNEL=="lcd[0-9]*" , ATTRS{idVendor}=="15c2" , SYMLINK+="lcd-imon" journalctl -k shows: Jun 02 18:36:11 frodo kernel: input: iMON Remote (15c2:0038) (lircd bypass) as /devices/virtual/input/input29 Jun 02 18:36:11 frodo kernel: input: lircd-uinput as /devices/virtual/input/input30 Jun 02 18:36:11 frodo kernel: r8169 0000:03:00.0 eth0: Link is Up - 1Gbps/Full - flow control rx/tx Jun 02 18:36:11 frodo kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Jun 02 18:36:58 frodo kernel: synth uevent: /devices/virtual/input/input30: failed to send uevent Jun 02 18:36:58 frodo kernel: input input30: uevent: failed to send synthetic uevent $ uname -a Linux frodo 5.6.0-2-amd64 #1 SMP Debian 5.6.14-1 (2020-05-23) x86_64 GNU/Linux Hope this helps
On second thought, this problem occurs when udevadm trigger is applied without restriction. I've no longer this problem when udevadm trigger is applied only on usbmisc subsystem. E.g.: udevadm trigger --action=change --subsystem-match=usbmisc Hope this helps
Is the device in question the Xen virtual keyboard? That's where I am seeing this and it causes the debian installer to crash and reboot ( init exits due to set -e plus udev error, causing kernel panic ) the domU when booting it under Xen.
Hi In my case, the device is an USB LCD display driven by lcdproc (see http://lcdproc.omnipotent.net/) All the best
I bisected the source of the problem to this commit: commit df44b479654f62b478c18ee4d8bc4e9f897a9844 Author: Peter Rajnoha <prajnoha@redhat.com> Date: Wed Dec 5 12:27:44 2018 +0100 kobject: return error code if writing /sys/.../uevent fails Propagate error code back to userspace if writing the /sys/.../uevent file fails. Before, the write operation always returned with success, even if we failed to recognize the input string or if we failed to generate the uevent itself. With the error codes properly propagated back to userspace, we are able to react in userspace accordingly by not assuming and awaiting a uevent that is not delivered. Signed-off-by: Peter Rajnoha <prajnoha@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> So I guess that certain drivers, including the Xen virtual keyboard and your LCD display, have always been broken but it was never reported until this commit.
So I have drilled down to the root cause finally. The modalias for this driver is over 2 KB in size, so the input core fails with -ENOMEM trying to shove it into the environment block for the uevent. I wonder if your device also has an unusually large modalias?
In my case, the failing lcdproc driver is imon. Here's the modalias size: $ grep 'imon$' /lib/modules/`uname -r`/modules.alias | wc -c 1020 HTH
This bug has a corresponding bug on Debian BTS: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=983357 This Debian bug is specifically about the Xen virtual keyboard driver. In that bug report, a Debian Developer noted that the environment of a uevent is limited to 2k in the kernel, but 8k is allocated in the systemd buffer for receiving uevents. So the root cause of the bug seems to be a mismatch in these buffer sizes in the kernel and systemd. Although this implies the buffer in the kernel should be increased to 8k, he suggested as a first step that someone test and see if increasing it to 4k helps. I did the test and it seems to fix the bug for the Xen virtual keyboard driver. The patch to the kernel to implement this fix was given by the Debian developer as: --- a/include/linux/kobject.h +++ b/include/linux/kobject.h @@ -30,7 +30,7 @@ #define UEVENT_HELPER_PATH_LEN 256 #define UEVENT_NUM_ENVP 64 /* number of env pointers */ -#define UEVENT_BUFFER_SIZE 2048 /* buffer for the variables */ +#define UEVENT_BUFFER_SIZE 4096 /* buffer for the variables */ #ifdef CONFIG_UEVENT_HELPER /* path to the userspace helper executed on an event */ --- END --- Anyone who sees this bug with a particular kernel driver might try this patch to see if it fixes it.