[mythtv-users] Very nasty system state

shadestalker at gmail.com shadestalker at gmail.com
Fri Aug 26 00:11:00 UTC 2005 

On 8/25/05, Brandon Beattie <brandon+myth at linuxis.us> wrote:
> Not sure if Myth is to blame, XFS, or something else but I thought I'd
> at least make a note incase it comes up again in the future.  Myth did
> start the cascade, but I think it was an XFS problem. Once
> this started, the system would lock a console on a sync, would not
> shut down, would not kill the mythbackend process even with -9.
> This is the first time I've ever seen this, running a cvs build from 2
> months ago, system hardware has been the same for 2 years.  No problems
> ever like this.  No software changes for 2 months at all.  After a hard
> reboot, XFS did some auto-fixing and it's running fine.  I've been
> running XFS for 3 years and never had single problem. ;)  No, not
> planning on changing either.
> 
> Comments?
> 
> --Brandon
> 
> [snip]
> 
> Aug 21 20:25:47 mindseye kernel: mythbackend: page allocation failure.  order:4, mode:0x50
> Aug 21 20:25:47 mindseye kernel:  [__alloc_pages+728/848] __alloc_pages+0x2d8/0x350
> Aug 21 20:25:47 mindseye kernel:  [__get_free_pages+37/64] __get_free_pages+0x25/0x40
> Aug 21 20:25:47 mindseye kernel:  [kmem_getpages+32/176] kmem_getpages+0x20/0xb0
> Aug 21 20:25:47 mindseye kernel:  [cache_grow+182/368] cache_grow+0xb6/0x170
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+2894788/6133658] xfs_alloc_delrec+0x111/0xf20 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [cache_alloc_refill+342/544] cache_alloc_refill+0x156/0x220
> Aug 21 20:25:47 mindseye kernel:  [__kmalloc+116/128] __kmalloc+0x74/0x80
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3291468/6133658] kmem_alloc+0x59/0xc0 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [unlock_page+31/48] unlock_page+0x1f/0x30
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3291758/6133658] kmem_realloc+0x2b/0xa0 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3149765/6133658] xfs_iext_realloc+0x112/0x160 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+2969240/6133658] xfs_bmap_insert_exlist+0x35/0x90 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+2954167/6133658] xfs_bmap_add_extent_hole_delay+0x394/0x5c0 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+2996568/6133658] xfs_bmbt_insrec+0x255/0x670 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+2942421/6133658] xfs_bmap_add_extent+0x412/0x4e0 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3223116/6133658] xfs_mod_incore_sb+0x39/0x60 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+2978022/6133658] xfs_bmapi+0x733/0x1600 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [buffered_rmqueue+245/464] buffered_rmqueue+0xf5/0x1d0
> Aug 21 20:25:47 mindseye kernel:  [__alloc_pages+746/848] __alloc_pages+0x2ea/0x350
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+2976860/6133658] xfs_bmapi+0x2a9/0x1600 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3163139/6133658] xfs_iomap_write_delay+0x520/0x6f0 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [as_add_request+422/528] as_add_request+0x1a6/0x210
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3158568/6133658] xfs_imap_to_bmap+0x35/0x2a0 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3240284/6133658] xfs_trans_unlocked_item+0x39/0x60 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3160024/6133658] xfs_iomap+0x345/0x550 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3297469/6133658] linvfs_get_block_core+0xaa/0x2b0 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [find_get_page+45/96] find_get_page+0x2d/0x60
> Aug 21 20:25:47 mindseye kernel:  [set_bh_page+72/80] set_bh_page+0x48/0x50
> Aug 21 20:25:47 mindseye kernel:  [create_buffers+106/176] create_buffers+0x6a/0xb0
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3298058/6133658] linvfs_get_block+0x47/0x50 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__block_prepare_write+518/1040] __block_prepare_write+0x206/0x410
> Aug 21 20:25:47 mindseye kernel:  [block_prepare_write+52/80] block_prepare_write+0x34/0x50
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3297987/6133658] linvfs_get_block+0x0/0x50 [xfs]
> Aug 21 20:25:47 mindseye kernel: [generic_file_aio_write_nolock+939/2864] generic_file_aio_write_nolock+0x3ab/0xb30
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3297987/6133658] linvfs_get_block+0x0/0x50 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+1605664/6133658] do_ide_request+0x1d/0x30 [ide_core]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3155725/6133658] xfs_ichgtime+0x10a/0x10c [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3240284/6133658] xfs_trans_unlocked_item+0x39/0x60 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3327333/6133658] xfs_write+0x272/0x890 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [schedule+654/1168] schedule+0x28e/0x490
> Aug 21 20:25:47 mindseye kernel:  [__crc_get_wchan+3308894/6133658] linvfs_write+0xab/0x120 [xfs]
> Aug 21 20:25:47 mindseye kernel:  [do_sync_write+132/176] do_sync_write+0x84/0xb0
> Aug 21 20:25:47 mindseye kernel:  [do_sync_write+0/176] do_sync_write+0x0/0xb0
> Aug 21 20:25:47 mindseye kernel:  [vfs_write+216/320] vfs_write+0xd8/0x140
> Aug 21 20:25:47 mindseye kernel:  [copy_to_user+62/80] copy_to_user+0x3e/0x50
> Aug 21 20:25:47 mindseye kernel:  [sys_write+81/128] sys_write+0x51/0x80
> Aug 21 20:25:47 mindseye kernel:  [syscall_call+7/11] syscall_call+0x7/0xb
> Aug 21 20:48:27 mindseye kernel: mythbackend: page allocation failure.  order:4, mode:0x50
> Aug 21 20:48:28 mindseye kernel:  [__alloc_pages+728/848] __alloc_pages+0x2d8/0x350
> Aug 21 20:48:28 mindseye kernel:  [__get_free_pages+37/64] __get_free_pages+0x25/0x40
> Aug 21 20:48:28 mindseye kernel:  [kmem_getpages+32/176] kmem_getpages+0x20/0xb0
> Aug 21 20:48:28 mindseye kernel:  [cache_grow+182/368] cache_grow+0xb6/0x170
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+1639347/6133658] __ide_dma_read+0xd0/0xe0 [ide_core]
> Aug 21 20:48:28 mindseye kernel:  [cache_alloc_refill+342/544] cache_alloc_refill+0x156/0x220
> Aug 21 20:48:28 mindseye kernel:  [__kmalloc+116/128] __kmalloc+0x74/0x80
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3291468/6133658] kmem_alloc+0x59/0xc0 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3142722/6133658] xfs_iread_extents+0x4f/0x110 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+2986294/6133658] xfs_bunmapi+0xff3/0x1090 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3236986/6133658] xfs_trans_unreserve_and_mod_sb+0x177/0x180 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3303673/6133658] pagebuf_rele+0x106/0x120 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3246046/6133658] xfs_trans_unlock_items+0x5b/0xe0 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3175812/6133658] xfs_log_release_iclog+0x21/0x60 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3185038/6133658] xlog_grant_log_space+0x12b/0x380 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3244912/6133658] xfs_trans_log_inode+0x2d/0x70 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3145044/6133658] xfs_itruncate_finish+0x1f1/0x460 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3269300/6133658] xfs_inactive+0x501/0x560 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3336537/6133658] vn_rele+0xc6/0xe0 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [__crc_get_wchan+3330795/6133658] linvfs_clear_inode+0x18/0x30 [xfs]
> Aug 21 20:48:28 mindseye kernel:  [clear_inode+210/256] clear_inode+0xd2/0x100
> Aug 21 20:48:28 mindseye kernel:  [generic_delete_inode+308/368] generic_delete_inode+0x134/0x170
> Aug 21 20:48:28 mindseye kernel:  [iput+98/128] iput+0x62/0x80
> Aug 21 20:48:28 mindseye kernel:  [sys_unlink+272/320] sys_unlink+0x110/0x140
> Aug 21 20:48:28 mindseye kernel:  [syscall_call+7/11] syscall_call+0x7/0xb
> Aug 21 20:48:28 mindseye kernel: mythbackend: page allocation failure.  order:4, mode:0x50
> Aug 21 20:48:28 mindseye kernel:  [__alloc_pages+728/848] __alloc_pages+0x2d8/0x350
> Aug 21 20:48:28 mindseye kernel:  [__get_free_pages+37/64] __get_free_pages+0x25/0x40
> 

I'd install smartmontools if it's not already and see what smartctl
says.  Get a baseline now and keep track of anything that changes for
the worse.

Have you tried a manual xfs_check from single-user mode?
If you do a controlled reboot and xfs_check, does it find things to
fix even though there wasn't a crash?
Any media errors at all, or just filesystem stuff?

More information about the mythtv-users mailing list