Linux: ALSA: 6fire: fix OOB write from device-reported iso length

In the Linux kernel, the following vulnerability has been resolved:

ALSA: 6fire: fix OOB write from device-reported iso length

usb6fire_pcm_in_urb_handler() sizes each outgoing isochronous packet as (actual_length - 4) / (in_n_analog << 2) * (out_n_analog << 2) + 4, where actual_length is the unsigned length the device reported for the matching IN packet. A packet completed with status 0 and actual_length < 4 wraps the subtraction to 0x7fffffec; a zero-length isochronous packet is legal on the bus, and the preceding loop rejects only non-zero status. The sum reaches memset() on out_urb->buffer, a 4832-byte object from kcalloc(PCM_MAX_PACKET_SIZE, PCM_N_PACKETS_PER_URB).

Even without the wrap the result is out of bounds: at 88.2/96 kHz the 4-in/6-out scaling turns a full 420-byte IN packet into 628, so eight packets span 5024 bytes of that buffer. usb_submit_urb() rejects an over-long descriptor only after the memset() and the usb6fire_pcm_playback() copy of user PCM data have run.

Guard the subtraction as the sibling usb6fire_pcm_capture() already does, and limit the frame count to what fits in rt->out_packet_size, the OUT endpoint's wMaxPacketSize. This bounds total_length by the buffer size while keeping each packet length aligned to a whole output frame.

BUG: KASAN: out-of-bounds in usb6fire_pcm_in_urb_handler (sound/usb/6fire/pcm.c:338) Write of size 18446744073709551456 at addr ffff88802a3d0000 by task vhci_rx/5018 Call Trace: dump_stack_lvl (lib/dump_stack.c:94 lib/dump_stack.c:120) print_report (mm/kasan/report.c:378 mm/kasan/report.c:482) kasan_report (mm/kasan/report.c:595) kasan_check_range (mm/kasan/generic.c:186 mm/kasan/generic.c:200) __asan_memset (mm/kasan/shadow.c:84) usb6fire_pcm_in_urb_handler (sound/usb/6fire/pcm.c:338) __usb_hcd_giveback_urb (drivers/usb/core/hcd.c:1657) usb_hcd_giveback_urb (drivers/usb/core/hcd.c:1741) vhci_rx_loop (drivers/usb/usbip/vhci_rx.c:107 drivers/usb/usbip/vhci_rx.c:242) kthread (kernel/kthread.c:436) ret_from_fork (arch/x86/kernel/process.c:158) ret_from_fork_asm (arch/x86/entry/entry_64.S:245)

Allocated by task 10: __kmalloc_cache_noprof (mm/slub.c:5563) usb6fire_pcm_init (sound/usb/6fire/pcm.c:560 sound/usb/6fire/pcm.c:595) usb6fire_chip_probe (sound/usb/6fire/chip.c:133) usb_probe_interface (drivers/usb/core/driver.c:399)

The buggy address belongs to the object at ffff88802a3d0000 which belongs to the cache kmalloc-8k of size 8192 The buggy address is located 0 bytes inside of 4832-byte region [ffff88802a3d0000, ffff88802a3d12e0) Kernel panic - not syncing: Fatal exception in interrupt