Begin forwarded message:

Date: Thu, 30 Mar 2006 05:32:16 -0800
From: bugme-daemon@bugzilla.kernel.org
To: bugme-new@lists.osdl.org
Subject: [Bugme-new] [Bug 6307] New: Bug hwclock in kernel 2.6.16


http://bugzilla.kernel.org/show_bug.cgi?id=6307

           Summary: Bug hwclock in kernel 2.6.16
    Kernel Version: 2.6.16
            Status: NEW
          Severity: normal
             Owner: johnstul@us.ibm.com
         Submitter: Gordeev@educom.ru


Most recent kernel where this bug did not occur: Kernel 2.6.8-2-itanium-smp 
(Defauld in Debian 3.1)
Distribution:
Hardware Environment: IBM xSeries 455 (Itanium 2)
Software Environment:Debian 3.1
Problem Description: running hwclock --hctosys or ntpdate $ip_addr_ntp_server 
hangup the kernel.

My kernel config:

# Automatically generated make config: don't edit
#

#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y

#
# General setup
#
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
CONFIG_LOG_BUF_SHIFT=16
CONFIG_HOTPLUG=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set

#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
CONFIG_OBSOLETE_MODPARM=y
CONFIG_MODVERSIONS=y
CONFIG_KMOD=y
CONFIG_STOP_MACHINE=y

#
# Processor type and features
#
CONFIG_IA64=y
CONFIG_64BIT=y
CONFIG_MMU=y
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_TIME_INTERPOLATION=y
CONFIG_EFI=y
CONFIG_IA64_GENERIC=y
# CONFIG_IA64_DIG is not set
# CONFIG_IA64_HP_ZX1 is not set
# CONFIG_IA64_SGI_SN2 is not set
# CONFIG_IA64_HP_SIM is not set
CONFIG_ITANIUM=y
# CONFIG_MCKINLEY is not set
# CONFIG_IA64_PAGE_SIZE_4KB is not set
# CONFIG_IA64_PAGE_SIZE_8KB is not set
CONFIG_IA64_PAGE_SIZE_16KB=y
# CONFIG_IA64_PAGE_SIZE_64KB is not set
CONFIG_IA64_BRL_EMU=y
# CONFIG_ITANIUM_BSTEP_SPECIFIC is not set
CONFIG_IA64_L1_CACHE_SHIFT=6
CONFIG_NUMA=y
CONFIG_VIRTUAL_MEM_MAP=y
CONFIG_DISCONTIGMEM=y
# CONFIG_IA64_CYCLONE is not set
CONFIG_IOSAPIC=y
CONFIG_FORCE_MAX_ZONEORDER=18
CONFIG_SMP=y
CONFIG_NR_CPUS=64
CONFIG_HOTPLUG_CPU=y
CONFIG_PREEMPT=y
CONFIG_HAVE_DEC_LOCK=y
CONFIG_IA32_SUPPORT=y
CONFIG_COMPAT=y
CONFIG_PERFMON=y
CONFIG_IA64_PALINFO=m

#
# Firmware Drivers
#
CONFIG_EFI_VARS=m
CONFIG_EFI_PCDP=y
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_MISC=m

#
# Power management and ACPI
#
CONFIG_PM=y
CONFIG_ACPI=y

#
# ACPI (Advanced Configuration and Power Interface) Support
#
CONFIG_ACPI_BOOT=y
CONFIG_ACPI_INTERPRETER=y
CONFIG_ACPI_BUTTON=m
CONFIG_ACPI_FAN=m
CONFIG_ACPI_PROCESSOR=m
CONFIG_ACPI_THERMAL=m
CONFIG_ACPI_NUMA=y
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_BUS=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_PCI=y
CONFIG_ACPI_SYSTEM=y

#
# Bus options (PCI, PCMCIA)
#
CONFIG_PCI=y
CONFIG_PCI_DOMAINS=y
CONFIG_PCI_MSI=y
# CONFIG_PCI_LEGACY_PROC is not set
CONFIG_PCI_NAMES=y

#
# PCI Hotplug Support
#
CONFIG_HOTPLUG_PCI=m
CONFIG_HOTPLUG_PCI_FAKE=m
CONFIG_HOTPLUG_PCI_ACPI=m
CONFIG_HOTPLUG_PCI_CPCI=y
CONFIG_HOTPLUG_PCI_PCIE=m
# CONFIG_HOTPLUG_PCI_PCIE_POLL_EVENT_MODE is not set
CONFIG_HOTPLUG_PCI_SHPC=m
# CONFIG_HOTPLUG_PCI_SHPC_POLL_EVENT_MODE is not set
#
# PCMCIA/CardBus support
#
CONFIG_PCMCIA=m
# CONFIG_PCMCIA_DEBUG is not set
CONFIG_YENTA=m
CONFIG_CARDBUS=y
CONFIG_PD6729=m
CONFIG_I82092=m
CONFIG_TCIC=m

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=m
# CONFIG_DEBUG_DRIVER is not set

#
# Memory Technology Devices (MTD)
#
CONFIG_MTD=m
# CONFIG_MTD_DEBUG is not set
CONFIG_MTD_PARTITIONS=y
CONFIG_MTD_CONCAT=m
CONFIG_MTD_REDBOOT_PARTS=m
# CONFIG_MTD_REDBOOT_PARTS_UNALLOCATED is not set
# CONFIG_MTD_REDBOOT_PARTS_READONLY is not set
# CONFIG_MTD_CMDLINE_PARTS is not set

#
# User Modules And Translation Layers
#
CONFIG_MTD_CHAR=m
CONFIG_MTD_BLOCK=m
CONFIG_MTD_BLOCK_RO=m
CONFIG_FTL=m
CONFIG_NFTL=m
CONFIG_NFTL_RW=y
CONFIG_INFTL=m

#
# RAM/ROM/Flash chip drivers
#
CONFIG_MTD_CFI=m
CONFIG_MTD_JEDECPROBE=m
CONFIG_MTD_GEN_PROBE=m
# CONFIG_MTD_CFI_ADV_OPTIONS is not set
CONFIG_MTD_MAP_BANK_WIDTH_1=y
CONFIG_MTD_MAP_BANK_WIDTH_2=y
CONFIG_MTD_MAP_BANK_WIDTH_4=y
# CONFIG_MTD_MAP_BANK_WIDTH_8 is not set
# CONFIG_MTD_MAP_BANK_WIDTH_16 is not set
# CONFIG_MTD_MAP_BANK_WIDTH_32 is not set
CONFIG_MTD_CFI_I1=y
CONFIG_MTD_CFI_I2=y
# CONFIG_MTD_CFI_I4 is not set
# CONFIG_MTD_CFI_I8 is not set
CONFIG_MTD_CFI_INTELEXT=m
CONFIG_MTD_CFI_AMDSTD=m
CONFIG_MTD_CFI_AMDSTD_RETRY=0
CONFIG_MTD_CFI_STAA=m
CONFIG_MTD_CFI_UTIL=m
CONFIG_MTD_RAM=m
CONFIG_MTD_ROM=m
CONFIG_MTD_ABSENT=m

#
# Mapping drivers for chip access
#
CONFIG_MTD_COMPLEX_MAPPINGS=y
CONFIG_MTD_PHYSMAP=m
CONFIG_MTD_PHYSMAP_START=0x8000000
CONFIG_MTD_PHYSMAP_LEN=0x4000000
CONFIG_MTD_PHYSMAP_BANKWIDTH=2
CONFIG_MTD_PCI=m

#
# Self-contained MTD device drivers
#
CONFIG_MTD_PMC551=m
# CONFIG_MTD_PMC551_BUGFIX is not set
# CONFIG_MTD_PMC551_DEBUG is not set
CONFIG_MTD_SLRAM=m
CONFIG_MTD_PHRAM=m
CONFIG_MTD_MTDRAM=m
CONFIG_MTDRAM_TOTAL_SIZE=4096
CONFIG_MTDRAM_ERASE_SIZE=128
CONFIG_MTD_BLKMTD=m

#
# Disk-On-Chip Device Drivers
#
CONFIG_MTD_DOC2000=m
CONFIG_MTD_DOC2001=m
CONFIG_MTD_DOC2001PLUS=m
CONFIG_MTD_DOCPROBE=m
CONFIG_MTD_DOCECC=m
# CONFIG_MTD_DOCPROBE_ADVANCED is not set
CONFIG_MTD_DOCPROBE_ADDRESS=0

#
# NAND Flash Device Drivers
#
CONFIG_MTD_NAND=m
# CONFIG_MTD_NAND_VERIFY_WRITE is not set
CONFIG_MTD_NAND_IDS=m
CONFIG_MTD_NAND_DISKONCHIP=m
# CONFIG_MTD_NAND_DISKONCHIP_PROBE_ADVANCED is not set
CONFIG_MTD_NAND_DISKONCHIP_PROBE_ADDRESS=0
# CONFIG_MTD_NAND_DISKONCHIP_BBTWRITE is not set

#
# Parallel port support
#
CONFIG_PARPORT=m
CONFIG_PARPORT_PC=m
CONFIG_PARPORT_PC_CML1=m
CONFIG_PARPORT_SERIAL=m
# CONFIG_PARPORT_PC_FIFO is not set
# CONFIG_PARPORT_PC_SUPERIO is not set
# CONFIG_PARPORT_PC_PCMCIA is not set
# CONFIG_PARPORT_OTHER is not set
CONFIG_PARPORT_1284=y

#
# Plug and Play support
#

#
# Block devices
#
CONFIG_PARIDE=m
CONFIG_PARIDE_PARPORT=m

#
# Parallel IDE high-level drivers
#
CONFIG_PARIDE_PD=m
CONFIG_PARIDE_PCD=m
CONFIG_PARIDE_PF=m
CONFIG_PARIDE_PT=m
CONFIG_PARIDE_PG=m

#
# Parallel IDE protocol modules
#
CONFIG_PARIDE_ATEN=m
CONFIG_PARIDE_BPCK=m
CONFIG_PARIDE_COMM=m
CONFIG_PARIDE_DSTR=m
CONFIG_PARIDE_FIT2=m
CONFIG_PARIDE_FIT3=m
CONFIG_PARIDE_EPAT=m
CONFIG_PARIDE_EPATC8=y
CONFIG_PARIDE_EPIA=m
CONFIG_PARIDE_FRIQ=m
CONFIG_PARIDE_FRPW=m
CONFIG_PARIDE_KBIC=m
CONFIG_PARIDE_KTTI=m
CONFIG_PARIDE_ON20=m
CONFIG_PARIDE_ON26=m
CONFIG_BLK_CPQ_DA=m
CONFIG_BLK_CPQ_CISS_DA=m
CONFIG_CISS_SCSI_TAPE=y
CONFIG_BLK_DEV_DAC960=m
CONFIG_BLK_DEV_UMEM=m
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_SX8=m
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=32768
CONFIG_BLK_DEV_INITRD=y

#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=m
CONFIG_BLK_DEV_IDE=m

#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_IDE_SATA is not set
CONFIG_BLK_DEV_IDEDISK=m
# CONFIG_IDEDISK_MULTI_MODE is not set
# CONFIG_BLK_DEV_IDECS is not set
CONFIG_BLK_DEV_IDECD=m
CONFIG_BLK_DEV_IDETAPE=m
CONFIG_BLK_DEV_IDEFLOPPY=m
CONFIG_BLK_DEV_IDESCSI=m
CONFIG_IDE_TASK_IOCTL=y
CONFIG_IDE_TASKFILE_IO=y

#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=m
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
# CONFIG_BLK_DEV_OFFBOARD is not set
CONFIG_BLK_DEV_GENERIC=m
CONFIG_BLK_DEV_OPTI621=m
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
CONFIG_BLK_DEV_ADMA=y
CONFIG_BLK_DEV_AEC62XX=m
CONFIG_BLK_DEV_ALI15X3=m
CONFIG_WDC_ALI15X3=y
CONFIG_BLK_DEV_AMD74XX=m
CONFIG_BLK_DEV_CMD64X=m
CONFIG_BLK_DEV_TRIFLEX=m
CONFIG_BLK_DEV_CY82C693=m
CONFIG_BLK_DEV_CS5520=m
CONFIG_BLK_DEV_CS5530=m
CONFIG_BLK_DEV_HPT34X=m
# CONFIG_HPT34X_AUTODMA is not set
CONFIG_BLK_DEV_HPT366=m
CONFIG_BLK_DEV_SC1200=m
CONFIG_BLK_DEV_PIIX=m
CONFIG_BLK_DEV_NS87415=m
CONFIG_BLK_DEV_PDC202XX_OLD=m
# CONFIG_PDC202XX_BURST is not set
CONFIG_BLK_DEV_PDC202XX_NEW=m
# CONFIG_PDC202XX_FORCE is not set
CONFIG_BLK_DEV_SVWKS=m
CONFIG_BLK_DEV_SGIIOC4=m
CONFIG_BLK_DEV_SIIMAGE=m
CONFIG_BLK_DEV_SLC90E66=m
CONFIG_BLK_DEV_TRM290=m
CONFIG_BLK_DEV_VIA82CXXX=m
# CONFIG_IDE_ARM is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set

#
# SCSI device support
#
CONFIG_SCSI=m
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=m
CONFIG_CHR_DEV_ST=m
CONFIG_CHR_DEV_OSST=m
CONFIG_BLK_DEV_SR=m
CONFIG_BLK_DEV_SR_VENDOR=y
CONFIG_CHR_DEV_SG=m
CONFIG_CHR_DEV_SCH=m

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y

#
# SCSI Transport Attributes
#
CONFIG_SCSI_SPI_ATTRS=m
CONFIG_SCSI_FC_ATTRS=m

#
# SCSI low-level drivers
#
CONFIG_BLK_DEV_3W_XXXX_RAID=m
CONFIG_SCSI_3W_9XXX=m
CONFIG_SCSI_ACARD=m
CONFIG_SCSI_AACRAID=m
CONFIG_SCSI_AIC7XXX=m
CONFIG_AIC7XXX_CMDS_PER_DEVICE=253
CONFIG_AIC7XXX_RESET_DELAY_MS=15000
# CONFIG_AIC7XXX_DEBUG_ENABLE is not set
CONFIG_AIC7XXX_DEBUG_MASK=0
# CONFIG_AIC7XXX_REG_PRETTY_PRINT is not set
CONFIG_SCSI_AIC7XXX_OLD=m
CONFIG_SCSI_AIC79XX=m
CONFIG_AIC79XX_CMDS_PER_DEVICE=32
CONFIG_AIC79XX_RESET_DELAY_MS=15000
# CONFIG_AIC79XX_ENABLE_RD_STRM is not set
# CONFIG_AIC79XX_DEBUG_ENABLE is not set
CONFIG_AIC79XX_DEBUG_MASK=0
# CONFIG_AIC79XX_REG_PRETTY_PRINT is not set
CONFIG_SCSI_MEGARAID=m
CONFIG_SCSI_SATA=y
# CONFIG_SCSI_SATA_SVW is not set
CONFIG_SCSI_ATA_PIIX=m
CONFIG_SCSI_SATA_NV=m
CONFIG_SCSI_SATA_PROMISE=m
CONFIG_SCSI_SATA_SX4=m
CONFIG_SCSI_SATA_SIL=m
CONFIG_SCSI_SATA_SIS=m
CONFIG_SCSI_SATA_VIA=m
CONFIG_SCSI_SATA_VITESSE=m
# CONFIG_SCSI_BUSLOGIC is not set
CONFIG_SCSI_DMX3191D=m
# CONFIG_SCSI_EATA is not set
CONFIG_SCSI_EATA_PIO=m
CONFIG_SCSI_FUTURE_DOMAIN=m
CONFIG_SCSI_GDTH=m
CONFIG_SCSI_IPS=m
CONFIG_SCSI_INIA100=m
CONFIG_SCSI_PPA=m
CONFIG_SCSI_IMM=m
# CONFIG_SCSI_IZIP_EPP16 is not set
# CONFIG_SCSI_IZIP_SLOW_CTR is not set
CONFIG_SCSI_SYM53C8XX_2=m
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=0
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
# CONFIG_SCSI_SYM53C8XX_IOMAPPED is not set
CONFIG_SCSI_IPR=m
# CONFIG_SCSI_IPR_TRACE is not set
# CONFIG_SCSI_IPR_DUMP is not set
CONFIG_SCSI_QLOGIC_ISP=m
CONFIG_SCSI_QLOGIC_FC=m
CONFIG_SCSI_QLOGIC_FC_FIRMWARE=y
CONFIG_SCSI_QLOGIC_1280=m
CONFIG_SCSI_QLOGIC_1280_1040=y
CONFIG_SCSI_DC395x=m
CONFIG_SCSI_DC390T=m
# CONFIG_SCSI_DEBUG is not set

#
# PCMCIA SCSI adapter support
#
CONFIG_PCMCIA_FDOMAIN=m
CONFIG_PCMCIA_QLOGIC=m
CONFIG_PCMCIA_SYM53C500=m

#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
CONFIG_BLK_DEV_MD=m
CONFIG_MD_LINEAR=m
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
CONFIG_MD_RAID5=m
CONFIG_MD_RAID6=m
CONFIG_MD_MULTIPATH=m
CONFIG_BLK_DEV_DM=m
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_MIRROR=m
CONFIG_DM_ZERO=m

#
# Fusion MPT device support
#
CONFIG_FUSION=m
CONFIG_FUSION_MAX_SGE=40
CONFIG_FUSION_ISENSE=m
CONFIG_FUSION_CTL=m
CONFIG_FUSION_LAN=m

#
# IEEE 1394 (FireWire) support
#
CONFIG_IEEE1394=m

#
# Subsystem Options
#
# CONFIG_IEEE1394_VERBOSEDEBUG is not set
CONFIG_IEEE1394_OUI_DB=y
CONFIG_IEEE1394_EXTRA_CONFIG_ROMS=y
CONFIG_IEEE1394_CONFIG_ROM_IP1394=y

#
# Device Drivers
#
CONFIG_IEEE1394_PCILYNX=m
CONFIG_IEEE1394_OHCI1394=m

#
# Protocol Drivers
#
CONFIG_IEEE1394_VIDEO1394=m
CONFIG_IEEE1394_SBP2=m
# CONFIG_IEEE1394_SBP2_PHYS_DMA is not set
CONFIG_IEEE1394_ETH1394=m
CONFIG_IEEE1394_DV1394=m
CONFIG_IEEE1394_RAWIO=m
CONFIG_IEEE1394_CMP=m
CONFIG_IEEE1394_AMDTP=m

#
# I2O device support
#
CONFIG_I2O=m
CONFIG_I2O_CONFIG=m
CONFIG_I2O_BLOCK=m
CONFIG_I2O_SCSI=m
CONFIG_I2O_PROC=m
#
# IP: Virtual Server Configuration
#
CONFIG_IP_VS=m
# CONFIG_IP_VS_DEBUG is not set
CONFIG_IP_VS_TAB_BITS=12

#
# IPVS transport protocol load balancing support
#
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_PROTO_ESP=y
CONFIG_IP_VS_PROTO_AH=y

#
# IPVS scheduler
#
CONFIG_IP_VS_RR=m
CONFIG_IP_VS_WRR=m
CONFIG_IP_VS_LC=m
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_LBLC=m
CONFIG_IP_VS_LBLCR=m
CONFIG_IP_VS_DH=m
CONFIG_IP_VS_SH=m
CONFIG_IP_VS_SED=m
CONFIG_IP_VS_NQ=m

#
# IPVS application helper
#
CONFIG_IP_VS_FTP=m
CONFIG_IPV6=m

#
# Networking support
#
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=y
# CONFIG_PACKET_MMAP is not set
CONFIG_NETLINK_DEV=m
CONFIG_UNIX=y
CONFIG_NET_KEY=m
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_FWMARK=y
CONFIG_IP_ROUTE_NAT=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_TOS=y
CONFIG_IP_ROUTE_VERBOSE=y
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=m
# CONFIG_NET_IPGRE is not set
# CONFIG_IP_MROUTE is not set
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
CONFIG_INET_IPCOMP=m
#
# IP: Virtual Server Configuration
#
CONFIG_IP_VS=m
# CONFIG_IP_VS_DEBUG is not set
CONFIG_IP_VS_TAB_BITS=12

#
# IPVS transport protocol load balancing support
#
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_PROTO_ESP=y
CONFIG_IP_VS_PROTO_AH=y

#
# IPVS scheduler
#
CONFIG_IP_VS_RR=m
CONFIG_IP_VS_WRR=m
CONFIG_IP_VS_LC=m
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_LBLC=m
CONFIG_IP_VS_LBLCR=m
CONFIG_IP_VS_DH=m
CONFIG_IP_VS_SH=m
CONFIG_IP_VS_SED=m
CONFIG_IP_VS_NQ=m

#
# IPVS application helper
#
CONFIG_IP_VS_FTP=m
CONFIG_IPV6=m
# CONFIG_IPV6_PRIVACY is not set
# CONFIG_INET6_AH is not set
# CONFIG_INET6_ESP is not set
CONFIG_INET6_IPCOMP=m
CONFIG_IPV6_TUNNEL=m
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
CONFIG_BRIDGE_NETFILTER=y

#
# IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=m
CONFIG_IP_NF_FTP=m
CONFIG_IP_NF_IRC=m
CONFIG_IP_NF_TFTP=m
CONFIG_IP_NF_AMANDA=m
CONFIG_IP_NF_QUEUE=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_LIMIT=m
CONFIG_IP_NF_MATCH_IPRANGE=m
CONFIG_IP_NF_MATCH_MAC=m
CONFIG_IP_NF_MATCH_PKTTYPE=m
CONFIG_IP_NF_MATCH_MARK=m
CONFIG_IP_NF_MATCH_MULTIPORT=m
CONFIG_IP_NF_MATCH_TOS=m
CONFIG_IP_NF_MATCH_RECENT=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_DSCP=m
CONFIG_IP_NF_MATCH_AH_ESP=m
CONFIG_IP_NF_MATCH_LENGTH=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_MATCH_TCPMSS=m
CONFIG_IP_NF_MATCH_HELPER=m
CONFIG_IP_NF_MATCH_STATE=m
CONFIG_IP_NF_MATCH_CONNTRACK=m
CONFIG_IP_NF_MATCH_OWNER=m
CONFIG_IP_NF_MATCH_PHYSDEV=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_NAT=m
CONFIG_IP_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_SAME=m
CONFIG_IP_NF_NAT_LOCAL=y
CONFIG_IP_NF_NAT_SNMP_BASIC=m
CONFIG_IP_NF_NAT_IRC=m
CONFIG_IP_NF_NAT_FTP=m
CONFIG_IP_NF_NAT_TFTP=m
CONFIG_IP_NF_NAT_AMANDA=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_TOS=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_DSCP=m
CONFIG_IP_NF_TARGET_MARK=m
CONFIG_IP_NF_TARGET_CLASSIFY=m
CONFIG_IP_NF_TARGET_LOG=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_IP_NF_TARGET_TCPMSS=m
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_IP_NF_COMPAT_IPCHAINS=m
CONFIG_IP_NF_COMPAT_IPFWADM=m
CONFIG_IP_NF_TARGET_NOTRACK=m
CONFIG_IP_NF_RAW=m
CONFIG_IP_NF_MATCH_ADDRTYPE=m
CONFIG_IP_NF_MATCH_REALM=m

#
# IPv6: Netfilter Configuration
#
# CONFIG_IP6_NF_QUEUE is not set
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_LIMIT=m
CONFIG_IP6_NF_MATCH_MAC=m
CONFIG_IP6_NF_MATCH_RT=m
CONFIG_IP6_NF_MATCH_OPTS=m
CONFIG_IP6_NF_MATCH_FRAG=m
CONFIG_IP6_NF_MATCH_HL=m
CONFIG_IP6_NF_MATCH_MULTIPORT=m
CONFIG_IP6_NF_MATCH_OWNER=m
CONFIG_IP6_NF_MATCH_MARK=m
CONFIG_IP6_NF_MATCH_IPV6HEADER=m
CONFIG_IP6_NF_MATCH_AHESP=m
CONFIG_IP6_NF_MATCH_LENGTH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_LOG=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_TARGET_MARK=m
CONFIG_IP6_NF_RAW=m

#
# DECnet: Netfilter Configuration
#
CONFIG_DECNET_NF_GRABULATOR=m

#
# Bridge: Netfilter Configuration
#
CONFIG_BRIDGE_NF_EBTABLES=m
CONFIG_BRIDGE_EBT_BROUTE=m
CONFIG_BRIDGE_EBT_T_FILTER=m
CONFIG_BRIDGE_EBT_T_NAT=m
CONFIG_BRIDGE_EBT_802_3=m
CONFIG_BRIDGE_EBT_AMONG=m
CONFIG_BRIDGE_EBT_ARP=m
CONFIG_BRIDGE_EBT_IP=m
CONFIG_BRIDGE_EBT_LIMIT=m
CONFIG_BRIDGE_EBT_MARK=m
CONFIG_BRIDGE_EBT_PKTTYPE=m
CONFIG_BRIDGE_EBT_STP=m
CONFIG_BRIDGE_EBT_VLAN=m
CONFIG_BRIDGE_EBT_ARPREPLY=m
CONFIG_BRIDGE_EBT_DNAT=m
CONFIG_BRIDGE_EBT_MARK_T=m
CONFIG_BRIDGE_EBT_REDIRECT=m
CONFIG_BRIDGE_EBT_SNAT=m
CONFIG_BRIDGE_EBT_LOG=m
CONFIG_XFRM=y
CONFIG_XFRM_USER=m

#
# SCTP Configuration (EXPERIMENTAL)
#
CONFIG_IP_SCTP=m
# CONFIG_SCTP_DBG_MSG is not set
# CONFIG_SCTP_DBG_OBJCNT is not set
CONFIG_SCTP_HMAC_NONE=y
# CONFIG_SCTP_HMAC_SHA1 is not set
# CONFIG_SCTP_HMAC_MD5 is not set
# CONFIG_ATM is not set
CONFIG_BRIDGE=m
CONFIG_VLAN_8021Q=m
CONFIG_DECNET=m
# CONFIG_DECNET_SIOCGIFCONF is not set
# CONFIG_DECNET_ROUTER is not set
CONFIG_LLC=y
CONFIG_LLC2=m
CONFIG_IPX=m
# CONFIG_IPX_INTERN is not set
CONFIG_ATALK=m
CONFIG_DEV_APPLETALK=y
CONFIG_IPDDP=m
CONFIG_IPDDP_ENCAP=y
CONFIG_IPDDP_DECAP=y
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
CONFIG_WAN_ROUTER=m
# CONFIG_NET_HW_FLOWCONTROL is not set

#
# QoS and/or fair queueing
#
CONFIG_NET_SCHED=y
CONFIG_NET_SCH_CLK_JIFFIES=y
# CONFIG_NET_SCH_CLK_GETTIMEOFDAY is not set
# CONFIG_NET_SCH_CLK_CPU is not set
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
CONFIG_NET_SCH_NETEM=m
CONFIG_NET_SCH_INGRESS=m
CONFIG_NET_QOS=y
CONFIG_NET_ESTIMATOR=y
CONFIG_NET_CLS=y
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
# CONFIG_CLS_U32_PERF is not set
# CONFIG_NET_CLS_IND is not set
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
# CONFIG_NET_CLS_ACT is not set
CONFIG_NET_CLS_POLICE=y

#
# Network testing
#
CONFIG_NET_PKTGEN=m
CONFIG_NETPOLL=y
CONFIG_NETPOLL_RX=y
CONFIG_NETPOLL_TRAP=y
CONFIG_NET_POLL_CONTROLLER=y
CONFIG_HAMRADIO=y

#
# Packet Radio protocols
#
CONFIG_AX25=m
CONFIG_AX25_DAMA_SLAVE=y
CONFIG_NETROM=m
CONFIG_ROSE=m

#
# AX.25 network device drivers
#
CONFIG_BPQETHER=m
CONFIG_BAYCOM_SER_FDX=m
CONFIG_BAYCOM_SER_HDX=m
CONFIG_BAYCOM_PAR=m
CONFIG_YAM=m
CONFIG_IRDA=m

#
# IrDA protocols
#
CONFIG_IRLAN=m
CONFIG_IRNET=m
CONFIG_IRCOMM=m
# CONFIG_IRDA_ULTRA is not set

#
# IrDA options
#
# CONFIG_IRDA_CACHE_LAST_LSAP is not set
# CONFIG_IRDA_FAST_RR is not set
# CONFIG_IRDA_DEBUG is not set

#
# Infrared-port device drivers
#

#
# SIR device drivers
#
CONFIG_IRTTY_SIR=m

#
# Dongle support
#
# CONFIG_DONGLE is not set

#
# Old SIR device drivers
#

#
# Old Serial dongle support
#

#
# FIR device drivers
#
CONFIG_USB_IRDA=m
CONFIG_SIGMATEL_FIR=m
# CONFIG_VLSI_FIR is not set
CONFIG_BT=m
CONFIG_BT_L2CAP=m
CONFIG_BT_SCO=m
CONFIG_BT_RFCOMM=m
# CONFIG_BT_RFCOMM_TTY is not set
CONFIG_BT_BNEP=m
# CONFIG_BT_BNEP_MC_FILTER is not set
# CONFIG_BT_BNEP_PROTO_FILTER is not set
CONFIG_BT_HIDP=m

#
# Bluetooth device drivers
#
CONFIG_BT_HCIUSB=m
# CONFIG_BT_HCIUSB_SCO is not set
CONFIG_BT_HCIUART=m
# CONFIG_BT_HCIUART_H4 is not set
# CONFIG_BT_HCIUART_BCSP is not set
CONFIG_BT_HCIBCM203X=m
CONFIG_BT_HCIBFUSB=m
# CONFIG_BT_HCIDTL1 is not set
# CONFIG_BT_HCIBT3C is not set
# CONFIG_BT_HCIBLUECARD is not set
# CONFIG_BT_HCIBTUART is not set
CONFIG_BT_HCIVHCI=m
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
CONFIG_BONDING=m
CONFIG_EQUALIZER=m
CONFIG_TUN=m
CONFIG_ETHERTAP=m

#
# ARCnet devices
#
CONFIG_ARCNET=m
CONFIG_ARCNET_1201=m
CONFIG_ARCNET_1051=m
CONFIG_ARCNET_RAW=m
CONFIG_ARCNET_COM90xx=m
CONFIG_ARCNET_COM90xxIO=m
CONFIG_ARCNET_RIM_I=m
CONFIG_ARCNET_COM20020=m
CONFIG_ARCNET_COM20020_PCI=m

#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
CONFIG_NET_VENDOR_3COM=y
CONFIG_VORTEX=m
CONFIG_TYPHOON=m

#
# Tulip family network device support
#
CONFIG_NET_TULIP=y
CONFIG_DE2104X=m
CONFIG_TULIP=m
# CONFIG_TULIP_MWI is not set
# CONFIG_TULIP_MMIO is not set
# CONFIG_TULIP_NAPI is not set
CONFIG_DE4X5=m
CONFIG_WINBOND_840=m
CONFIG_DM9102=m
# CONFIG_PCMCIA_XIRCOM is not set
CONFIG_HP100=m
CONFIG_NET_PCI=y
CONFIG_PCNET32=m
CONFIG_AMD8111_ETH=m
# CONFIG_AMD8111E_NAPI is not set
CONFIG_ADAPTEC_STARFIRE=m
# CONFIG_ADAPTEC_STARFIRE_NAPI is not set
CONFIG_B44=m
CONFIG_FORCEDETH=m
CONFIG_EEPRO100=m
# CONFIG_EEPRO100_PIO is not set
CONFIG_E100=m
# CONFIG_E100_NAPI is not set
CONFIG_FEALNX=m
CONFIG_NATSEMI=m
CONFIG_NE2K_PCI=m
CONFIG_8139CP=m
CONFIG_8139TOO=m
# CONFIG_8139TOO_PIO is not set
# CONFIG_8139TOO_TUNE_TWISTER is not set
# CONFIG_8139TOO_8129 is not set
# CONFIG_8139_OLD_RX_RESET is not set
CONFIG_SIS900=m
CONFIG_EPIC100=m
CONFIG_SUNDANCE=m
# CONFIG_SUNDANCE_MMIO is not set
CONFIG_VIA_RHINE=m
# CONFIG_VIA_RHINE_MMIO is not set
CONFIG_VIA_VELOCITY=m

#
# Ethernet (1000 Mbit)
#
CONFIG_DL2K=m
CONFIG_E1000=m
CONFIG_E1000_NAPI=y
CONFIG_NS83820=m
CONFIG_HAMACHI=m
CONFIG_YELLOWFIN=m
CONFIG_R8169=m
CONFIG_SK98LIN=m
CONFIG_TIGON3=m

#
# Ethernet (10000 Mbit)
#
CONFIG_IXGB=m
# CONFIG_IXGB_NAPI is not set
CONFIG_S2IO=m
# CONFIG_S2IO_NAPI is not set

#
# Token Ring devices
#
CONFIG_TR=y
CONFIG_IBMOL=m
CONFIG_3C359=m
# CONFIG_TMS380TR is not set

#
# Wireless LAN (non-hamradio)
#
CONFIG_NET_RADIO=y

#
# Obsolete Wireless cards support (pre-802.11)
#
CONFIG_STRIP=m
# CONFIG_PCMCIA_WAVELAN is not set
# CONFIG_PCMCIA_NETWAVE is not set

#
# Wireless 802.11 Frequency Hopping cards support
#
# CONFIG_PCMCIA_RAYCS is not set

#
# Wireless 802.11b ISA/PCI cards support
#
CONFIG_HERMES=m
CONFIG_PLX_HERMES=m
CONFIG_TMD_HERMES=m
CONFIG_PCI_HERMES=m
CONFIG_ATMEL=m
CONFIG_PCI_ATMEL=m

#
# Wireless 802.11b Pcmcia/Cardbus cards support
#
# CONFIG_PCMCIA_HERMES is not set
# CONFIG_AIRO_CS is not set
CONFIG_PCMCIA_ATMEL=m
# CONFIG_PCMCIA_WL3501 is not set

#
# Prism GT/Duette 802.11(a/b/g) PCI/Cardbus support
#
CONFIG_PRISM54=m
CONFIG_NET_WIRELESS=y

#
# PCMCIA network device support
#
# CONFIG_NET_PCMCIA is not set

#
# Wan interfaces
#
CONFIG_WAN=y
CONFIG_DSCC4=m
# CONFIG_DSCC4_PCISYNC is not set
# CONFIG_DSCC4_PCI_RST is not set
CONFIG_LANMEDIA=m
CONFIG_SYNCLINK_SYNCPPP=m
CONFIG_HDLC=m
CONFIG_HDLC_RAW=y
CONFIG_HDLC_RAW_ETH=y
CONFIG_HDLC_CISCO=y
CONFIG_HDLC_FR=y
CONFIG_HDLC_PPP=y

#
# X.25/LAPB support is disabled
#
CONFIG_PCI200SYN=m
CONFIG_WANXL=m
CONFIG_PC300=m
CONFIG_PC300_MLPPP=y
CONFIG_FARSYNC=m
CONFIG_DLCI=m
CONFIG_DLCI_COUNT=24
CONFIG_DLCI_MAX=8
CONFIG_WAN_ROUTER_DRIVERS=y
# CONFIG_CYCLADES_SYNC is not set
CONFIG_FDDI=y
CONFIG_DEFXX=m
CONFIG_SKFP=m
# CONFIG_HIPPI is not set
CONFIG_PLIP=m
CONFIG_PPP=m
CONFIG_PPP_MULTILINK=y
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_BSDCOMP=m
CONFIG_PPPOE=m
CONFIG_SLIP=m
CONFIG_SLIP_COMPRESSED=y
CONFIG_SLIP_SMART=y
CONFIG_SLIP_MODE_SLIP6=y
CONFIG_NET_FC=y
CONFIG_SHAPER=m
CONFIG_NETCONSOLE=m

#
# ISDN subsystem
#
# CONFIG_ISDN is not set

#
# Telephony Support
#
CONFIG_PHONE=m
CONFIG_PHONE_IXJ=m
CONFIG_PHONE_IXJ_PCMCIA=m

#
# Input device support
#
CONFIG_INPUT=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_MOUSEDEV_PSAUX_ENABLE=y
CONFIG_INPUT_JOYDEV=m
CONFIG_INPUT_TSDEV=m
CONFIG_INPUT_TSDEV_SCREEN_X=240
CONFIG_INPUT_TSDEV_SCREEN_Y=320
CONFIG_INPUT_EVDEV=m
CONFIG_INPUT_EVBUG=m

#
# Input I/O drivers
#
CONFIG_GAMEPORT=m
CONFIG_SOUND_GAMEPORT=m
CONFIG_GAMEPORT_NS558=m
CONFIG_GAMEPORT_L4=m
CONFIG_GAMEPORT_EMU10K1=m
CONFIG_GAMEPORT_VORTEX=m
CONFIG_GAMEPORT_FM801=m
CONFIG_GAMEPORT_CS461x=m
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=m
CONFIG_SERIO_CT82C710=m
CONFIG_SERIO_PARKBD=m
CONFIG_SERIO_PCIPS2=m

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
CONFIG_KEYBOARD_SUNKBD=m
CONFIG_KEYBOARD_LKKBD=m
CONFIG_KEYBOARD_XTKBD=m
CONFIG_KEYBOARD_NEWTON=m
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=m
CONFIG_MOUSE_SERIAL=m
CONFIG_MOUSE_VSXXXAA=m
CONFIG_INPUT_JOYSTICK=y
CONFIG_JOYSTICK_ANALOG=m
CONFIG_JOYSTICK_A3D=m
CONFIG_JOYSTICK_ADI=m
CONFIG_JOYSTICK_COBRA=m
CONFIG_JOYSTICK_GF2K=m
CONFIG_JOYSTICK_GRIP=m
CONFIG_JOYSTICK_GRIP_MP=m
CONFIG_JOYSTICK_GUILLEMOT=m
CONFIG_JOYSTICK_INTERACT=m
CONFIG_JOYSTICK_SIDEWINDER=m
CONFIG_JOYSTICK_TMDC=m
CONFIG_JOYSTICK_IFORCE=m
# CONFIG_JOYSTICK_IFORCE_USB is not set
# CONFIG_JOYSTICK_IFORCE_232 is not set
CONFIG_JOYSTICK_WARRIOR=m
CONFIG_JOYSTICK_MAGELLAN=m
CONFIG_JOYSTICK_SPACEORB=m
CONFIG_JOYSTICK_SPACEBALL=m
CONFIG_JOYSTICK_STINGER=m
CONFIG_JOYSTICK_TWIDDLER=m
CONFIG_JOYSTICK_DB9=m
CONFIG_JOYSTICK_GAMECON=m
CONFIG_JOYSTICK_TURBOGRAFX=m
CONFIG_INPUT_JOYDUMP=m
# CONFIG_INPUT_TOUCHSCREEN is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_UINPUT=m

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_SERIAL_NONSTANDARD=y
CONFIG_ROCKETPORT=m
# CONFIG_CYCLADES is not set
# CONFIG_SYNCLINK is not set
CONFIG_SYNCLINKMP=m
CONFIG_N_HDLC=m
CONFIG_STALDRV=y

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_CS=m
CONFIG_SERIAL_8250_ACPI=y
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_SHARE_IRQ=y
# CONFIG_SERIAL_8250_DETECT_IRQ is not set
CONFIG_SERIAL_8250_MULTIPORT=y
CONFIG_SERIAL_8250_RSA=y

#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_SERIAL_SGI_L1_CONSOLE=y
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
CONFIG_PRINTER=m
# CONFIG_LP_CONSOLE is not set
CONFIG_PPDEV=m
CONFIG_TIPAR=m
# CONFIG_QIC02_TAPE is not set

#
# IPMI
#
CONFIG_IPMI_HANDLER=m
# CONFIG_IPMI_PANIC_EVENT is not set
CONFIG_IPMI_DEVICE_INTERFACE=m
CONFIG_IPMI_SI=m
CONFIG_IPMI_WATCHDOG=m

#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
# CONFIG_HW_RANDOM is not set
CONFIG_EFI_RTC=y
CONFIG_DTLK=m
CONFIG_R3964=m
CONFIG_APPLICOM=m

#
# Ftape, the floppy tape device driver
#
CONFIG_AGP=m
CONFIG_AGP_I460=m
CONFIG_AGP_HP_ZX1=m
CONFIG_DRM=y
CONFIG_DRM_TDFX=m
CONFIG_DRM_GAMMA=m
CONFIG_DRM_R128=m
CONFIG_DRM_RADEON=m
CONFIG_DRM_MGA=m
CONFIG_DRM_SIS=m

#
# PCMCIA character devices
#
# CONFIG_SYNCLINK_CS is not set
CONFIG_RAW_DRIVER=m
# CONFIG_HPET is not set
CONFIG_MAX_RAW_DEVS=256

#
# I2C support
#
CONFIG_I2C=m
CONFIG_I2C_CHARDEV=m

#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCF=m

#
# I2C Hardware Bus support
#
CONFIG_I2C_ALI1535=m
CONFIG_I2C_ALI1563=m
CONFIG_I2C_ALI15X3=m
CONFIG_I2C_AMD756=m
CONFIG_I2C_AMD8111=m
CONFIG_I2C_I801=m
CONFIG_I2C_I810=m
CONFIG_I2C_ISA=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_PARPORT=m
CONFIG_I2C_PARPORT_LIGHT=m
CONFIG_I2C_PROSAVAGE=m
CONFIG_I2C_SAVAGE4=m
CONFIG_SCx200_ACB=m
CONFIG_I2C_SIS5595=m
CONFIG_I2C_SIS630=m
CONFIG_I2C_SIS96X=m
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m
CONFIG_I2C_VOODOO3=m

#
# Hardware Sensors Chip support
#
CONFIG_I2C_SENSOR=m
CONFIG_SENSORS_ADM1021=m
CONFIG_SENSORS_ADM1025=m
CONFIG_SENSORS_ADM1031=m
CONFIG_SENSORS_ASB100=m
CONFIG_SENSORS_DS1621=m
CONFIG_SENSORS_FSCHER=m
CONFIG_SENSORS_GL518SM=m
CONFIG_SENSORS_IT87=m
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
CONFIG_SENSORS_LM78=m
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
CONFIG_SENSORS_LM90=m
CONFIG_SENSORS_MAX1619=m
CONFIG_SENSORS_VIA686A=m
CONFIG_SENSORS_W83781D=m
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83627HF=m

#
# Other I2C Chip support
#
CONFIG_SENSORS_EEPROM=m
CONFIG_SENSORS_PCF8574=m
CONFIG_SENSORS_PCF8591=m
CONFIG_SENSORS_RTC8564=m
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set

#
# Dallas's 1-wire bus
#
CONFIG_W1=m
CONFIG_W1_MATROX=m
CONFIG_W1_THERM=m

#
# Misc devices
#

#
# Multimedia devices
#
CONFIG_VIDEO_DEV=m

#
# Video For Linux
#

#
# Video Adapters
#
CONFIG_VIDEO_BT848=m
CONFIG_VIDEO_BWQCAM=m
CONFIG_VIDEO_CQCAM=m
CONFIG_VIDEO_W9966=m
CONFIG_VIDEO_CPIA=m
CONFIG_VIDEO_CPIA_PP=m
CONFIG_VIDEO_CPIA_USB=m
CONFIG_VIDEO_SAA5246A=m
CONFIG_VIDEO_SAA5249=m
CONFIG_TUNER_3036=m
CONFIG_VIDEO_STRADIS=m
CONFIG_VIDEO_ZORAN=m
CONFIG_VIDEO_ZORAN_BUZ=m
CONFIG_VIDEO_ZORAN_DC10=m
CONFIG_VIDEO_ZORAN_DC30=m
CONFIG_VIDEO_ZORAN_LML33=m
CONFIG_VIDEO_ZORAN_LML33R10=m
CONFIG_VIDEO_SAA7134=m
CONFIG_VIDEO_MXB=m
CONFIG_VIDEO_DPC=m
CONFIG_VIDEO_HEXIUM_ORION=m
CONFIG_VIDEO_HEXIUM_GEMINI=m
CONFIG_VIDEO_CX88=m
CONFIG_VIDEO_OVCAMCHIP=m

#
# Radio Adapters
#
CONFIG_RADIO_GEMTEK_PCI=m
CONFIG_RADIO_MAXIRADIO=m
CONFIG_RADIO_MAESTRO=m

#
# Digital Video Broadcasting Devices
#
CONFIG_DVB=y
CONFIG_DVB_CORE=m

#
# Supported Frontend Modules
#
CONFIG_DVB_TWINHAN_DST=m
CONFIG_DVB_STV0299=m
CONFIG_DVB_SP887X=m
CONFIG_DVB_SP887X_FIRMWARE_FILE="/usr/lib/hotplug/firmware/sc_main.mc"
CONFIG_DVB_ALPS_TDLB7=m
CONFIG_DVB_ALPS_TDMB7=m
CONFIG_DVB_ATMEL_AT76C651=m
CONFIG_DVB_CX24110=m
CONFIG_DVB_GRUNDIG_29504_491=m
CONFIG_DVB_GRUNDIG_29504_401=m
CONFIG_DVB_MT312=m
CONFIG_DVB_VES1820=m
CONFIG_DVB_VES1X93=m
CONFIG_DVB_TDA1004X=m
CONFIG_DVB_TDA1004X_FIRMWARE_FILE="/usr/lib/hotplug/firmware/tda1004x.bin"
CONFIG_DVB_NXT6000=m

#
# Supported SAA7146 based PCI Adapters
#
CONFIG_DVB_AV7110=m
# CONFIG_DVB_AV7110_OSD is not set
CONFIG_DVB_BUDGET=m
CONFIG_DVB_BUDGET_CI=m
CONFIG_DVB_BUDGET_AV=m
CONFIG_DVB_BUDGET_PATCH=m

#
# Supported USB Adapters
#
CONFIG_DVB_TTUSB_BUDGET=m
CONFIG_DVB_TTUSB_DEC=m

#
# Supported FlexCopII (B2C2) Adapters
#
CONFIG_DVB_B2C2_SKYSTAR=m

#
# Supported BT878 Adapters
#
CONFIG_DVB_BT8XX=m
CONFIG_VIDEO_SAA7146=m
CONFIG_VIDEO_SAA7146_VV=m
CONFIG_VIDEO_VIDEOBUF=m
CONFIG_VIDEO_TUNER=m
CONFIG_VIDEO_BUF=m
CONFIG_VIDEO_BTCX=m
CONFIG_VIDEO_IR=m

#
# Graphics support
#
CONFIG_FB=y
CONFIG_FB_CIRRUS=m
CONFIG_FB_PM2=m
# CONFIG_FB_PM2_FIFO_DISCONNECT is not set
CONFIG_FB_CYBER2000=m
CONFIG_FB_ASILIANT=y
# CONFIG_FB_IMSTT is not set
CONFIG_FB_RIVA=m
CONFIG_FB_RIVA_I2C=y
# CONFIG_FB_RIVA_DEBUG is not set
CONFIG_FB_MATROX=m
CONFIG_FB_MATROX_MILLENIUM=y
CONFIG_FB_MATROX_MYSTIQUE=y
CONFIG_FB_MATROX_G450=y
CONFIG_FB_MATROX_G100=y
CONFIG_FB_MATROX_I2C=m
CONFIG_FB_MATROX_MAVEN=m
CONFIG_FB_MATROX_MULTIHEAD=y
CONFIG_FB_RADEON_OLD=m
CONFIG_FB_RADEON=m
CONFIG_FB_RADEON_I2C=y
# CONFIG_FB_RADEON_DEBUG is not set
CONFIG_FB_ATY128=m
CONFIG_FB_ATY=m
CONFIG_FB_ATY_CT=y
CONFIG_FB_ATY_GX=y
# CONFIG_FB_ATY_XL_INIT is not set
CONFIG_FB_SIS=m
CONFIG_FB_SIS_300=y
CONFIG_FB_SIS_315=y
CONFIG_FB_NEOMAGIC=m
CONFIG_FB_KYRO=m
CONFIG_FB_3DFX=m
# CONFIG_FB_3DFX_ACCEL is not set
CONFIG_FB_VOODOO1=m
CONFIG_FB_TRIDENT=m
# CONFIG_FB_TRIDENT_ACCEL is not set
# CONFIG_FB_VIRTUAL is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_MDA_CONSOLE=m
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=m
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y

#
# Logo configuration
#
CONFIG_LOGO=y
CONFIG_LOGO_LINUX_MONO=y
CONFIG_LOGO_LINUX_VGA16=y
CONFIG_LOGO_LINUX_CLUT224=y

#
# Sound
#
CONFIG_SOUND=y

#
# Advanced Linux Sound Architecture
#
CONFIG_SND=m
CONFIG_SND_TIMER=m
CONFIG_SND_PCM=m
CONFIG_SND_HWDEP=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_SEQUENCER=m
CONFIG_SND_SEQ_DUMMY=m
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=m
CONFIG_SND_PCM_OSS=m
CONFIG_SND_SEQUENCER_OSS=y
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set

#
# Generic devices
#
CONFIG_SND_MPU401_UART=m
CONFIG_SND_OPL3_LIB=m
CONFIG_SND_VX_LIB=m
CONFIG_SND_DUMMY=m
CONFIG_SND_VIRMIDI=m
CONFIG_SND_MTPAV=m
CONFIG_SND_SERIAL_U16550=m
CONFIG_SND_MPU401=m

#
# PCI devices
#
CONFIG_SND_AC97_CODEC=m
CONFIG_SND_ALI5451=m
CONFIG_SND_ATIIXP=m
CONFIG_SND_AU8810=m
CONFIG_SND_AU8820=m
CONFIG_SND_AU8830=m
CONFIG_SND_AZT3328=m
CONFIG_SND_BT87X=m
CONFIG_SND_CS46XX=m
CONFIG_SND_CS46XX_NEW_DSP=y
CONFIG_SND_CS4281=m
CONFIG_SND_EMU10K1=m
CONFIG_SND_KORG1212=m
CONFIG_SND_MIXART=m
CONFIG_SND_NM256=m
CONFIG_SND_RME32=m
CONFIG_SND_RME96=m
CONFIG_SND_RME9652=m
CONFIG_SND_HDSP=m
CONFIG_SND_TRIDENT=m
CONFIG_SND_YMFPCI=m
CONFIG_SND_ALS4000=m
CONFIG_SND_CMIPCI=m
CONFIG_SND_ENS1370=m
CONFIG_SND_ENS1371=m
CONFIG_SND_ES1938=m
CONFIG_SND_ES1968=m
CONFIG_SND_MAESTRO3=m
CONFIG_SND_FM801=m
CONFIG_SND_FM801_TEA575X=m
CONFIG_SND_ICE1712=m
CONFIG_SND_ICE1724=m
CONFIG_SND_INTEL8X0=m
CONFIG_SND_INTEL8X0M=m
CONFIG_SND_SONICVIBES=m
CONFIG_SND_VIA82XX=m
CONFIG_SND_VX222=m

#
# ALSA USB devices
#
CONFIG_SND_USB_AUDIO=m

#
# PCMCIA devices
#

#
# Open Sound System
#
CONFIG_SOUND_PRIME=m
CONFIG_SOUND_BT878=m
CONFIG_SOUND_CMPCI=m
CONFIG_SOUND_EMU10K1=m
# CONFIG_MIDI_EMU10K1 is not set
CONFIG_SOUND_FUSION=m
CONFIG_SOUND_CS4281=m
CONFIG_SOUND_ES1370=m
CONFIG_SOUND_ES1371=m
CONFIG_SOUND_ESSSOLO1=m
CONFIG_SOUND_MAESTRO=m
CONFIG_SOUND_MAESTRO3=m
CONFIG_SOUND_ICH=m
CONFIG_SOUND_SONICVIBES=m
CONFIG_SOUND_TRIDENT=m
CONFIG_SOUND_MSNDCLAS=m
CONFIG_MSNDCLAS_INIT_FILE="m"
CONFIG_MSNDCLAS_PERM_FILE="m"
CONFIG_SOUND_MSNDPIN=m
CONFIG_MSNDPIN_INIT_FILE="m"
CONFIG_MSNDPIN_PERM_FILE="m"
CONFIG_SOUND_VIA82CXXX=m
# CONFIG_MIDI_VIA82CXXX is not set
# CONFIG_SOUND_OSS is not set
CONFIG_SOUND_TVMIXER=m
CONFIG_SOUND_ALI5455=m
CONFIG_SOUND_FORTE=m
CONFIG_SOUND_RME96XX=m
CONFIG_SOUND_AD1980=m

#
# USB support
#
CONFIG_USB=m
# CONFIG_USB_DEBUG is not set

#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
CONFIG_USB_BANDWIDTH=y
# CONFIG_USB_DYNAMIC_MINORS is not set

#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=m
# CONFIG_USB_EHCI_SPLIT_ISO is not set
CONFIG_USB_EHCI_ROOT_HUB_TT=y
CONFIG_USB_OHCI_HCD=m
CONFIG_USB_UHCI_HCD=m

#
# USB Device Class drivers
#
CONFIG_USB_AUDIO=m

#
# USB Bluetooth TTY can only be used with disabled Bluetooth subsystem
#
CONFIG_USB_MIDI=m
CONFIG_USB_ACM=m
CONFIG_USB_PRINTER=m
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
CONFIG_USB_STORAGE_RW_DETECT=y
CONFIG_USB_STORAGE_DATAFAB=y
CONFIG_USB_STORAGE_FREECOM=y
CONFIG_USB_STORAGE_ISD200=y
CONFIG_USB_STORAGE_DPCM=y
CONFIG_USB_STORAGE_HP8200e=y
CONFIG_USB_STORAGE_SDDR09=y
CONFIG_USB_STORAGE_SDDR55=y
CONFIG_USB_STORAGE_JUMPSHOT=y

#
# USB Human Interface Devices (HID)
#
CONFIG_USB_HID=m
CONFIG_USB_HIDINPUT=y
CONFIG_HID_FF=y
# CONFIG_HID_PID is not set
# CONFIG_LOGITECH_FF is not set
# CONFIG_THRUSTMASTER_FF is not set
CONFIG_USB_HIDDEV=y

#
# USB HID Boot Protocol drivers
#
CONFIG_USB_KBD=m
CONFIG_USB_MOUSE=m
CONFIG_USB_AIPTEK=m
CONFIG_USB_WACOM=m
CONFIG_USB_KBTAB=m
CONFIG_USB_POWERMATE=m
CONFIG_USB_MTOUCH=m
CONFIG_USB_EGALAX=m
CONFIG_USB_XPAD=m
CONFIG_USB_ATI_REMOTE=m

#
# USB Imaging devices
#
CONFIG_USB_MDC800=m
CONFIG_USB_MICROTEK=m
CONFIG_USB_HPUSBSCSI=m

#
# USB Multimedia devices
#
CONFIG_USB_VICAM=m
CONFIG_USB_DSBR=m
CONFIG_USB_IBMCAM=m
CONFIG_USB_KONICAWC=m
CONFIG_USB_OV511=m
CONFIG_USB_PWC=m
CONFIG_USB_SE401=m
CONFIG_USB_SN9C102=m
CONFIG_USB_STV680=m
CONFIG_USB_W9968CF=m

#
# USB Network adaptors
#
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
CONFIG_USB_USBNET=m

#
# USB Host-to-Host Cables
#
CONFIG_USB_ALI_M5632=y
CONFIG_USB_AN2720=y
CONFIG_USB_BELKIN=y
CONFIG_USB_GENESYS=y
CONFIG_USB_NET1080=y
CONFIG_USB_PL2301=y

#
# Intelligent USB Devices/Gadgets
#
CONFIG_USB_ARMLINUX=y
CONFIG_USB_EPSON2888=y
CONFIG_USB_ZAURUS=y
CONFIG_USB_CDCETHER=y

#
# USB Network Adapters
#
CONFIG_USB_AX8817X=y

#
# USB port drivers
#
CONFIG_USB_USS720=m

#
# USB Serial Converter support
#
CONFIG_USB_SERIAL=m
CONFIG_USB_SERIAL_GENERIC=y
CONFIG_USB_SERIAL_BELKIN=m
CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m
CONFIG_USB_SERIAL_EMPEG=m
CONFIG_USB_SERIAL_FTDI_SIO=m
CONFIG_USB_SERIAL_VISOR=m
CONFIG_USB_SERIAL_IPAQ=m
CONFIG_USB_SERIAL_IR=m
CONFIG_USB_SERIAL_EDGEPORT=m
CONFIG_USB_SERIAL_EDGEPORT_TI=m
CONFIG_USB_SERIAL_KEYSPAN_PDA=m
CONFIG_USB_SERIAL_KEYSPAN=m
CONFIG_USB_SERIAL_KLSI=m
CONFIG_USB_SERIAL_KOBIL_SCT=m
CONFIG_USB_SERIAL_MCT_U232=m
CONFIG_USB_SERIAL_PL2303=m
CONFIG_USB_SERIAL_SAFE=m
# CONFIG_USB_SERIAL_SAFE_PADDED is not set
CONFIG_USB_SERIAL_CYBERJACK=m
CONFIG_USB_SERIAL_XIRCOM=m
CONFIG_USB_SERIAL_OMNINET=m
CONFIG_USB_EZUSB=y

#
# USB Miscellaneous drivers
#
CONFIG_USB_TIGL=m
CONFIG_USB_AUERSWALD=m
CONFIG_USB_RIO500=m
CONFIG_USB_LEGOTOWER=m
CONFIG_USB_LCD=m
CONFIG_USB_LED=m
CONFIG_USB_CYTHERM=m
CONFIG_USB_PHIDGETSERVO=m
CONFIG_USB_TEST=m

#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set

#
# File systems
#
CONFIG_EXT2_FS=m
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
CONFIG_EXT3_FS=m
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_JBD=m
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=m
CONFIG_REISERFS_FS=m
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
CONFIG_REISERFS_FS_XATTR=y
CONFIG_REISERFS_FS_POSIX_ACL=y
CONFIG_REISERFS_FS_SECURITY=y
# CONFIG_JFS_FS is not set
CONFIG_FS_POSIX_ACL=y
CONFIG_XFS_FS=m
# CONFIG_XFS_RT is not set
CONFIG_XFS_QUOTA=y
CONFIG_XFS_SECURITY=y
CONFIG_XFS_POSIX_ACL=y
CONFIG_MINIX_FS=m
CONFIG_ROMFS_FS=m
CONFIG_QUOTA=y
CONFIG_QFMT_V1=m
CONFIG_QFMT_V2=m
CONFIG_QUOTACTL=y
CONFIG_AUTOFS_FS=m
CONFIG_AUTOFS4_FS=m

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_ZISOFS_FS=m
CONFIG_UDF_FS=m
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
CONFIG_DEVFS_FS=y
# CONFIG_DEVFS_MOUNT is not set
# CONFIG_DEVFS_DEBUG is not set
CONFIG_DEVPTS_FS_XATTR=y
CONFIG_DEVPTS_FS_SECURITY=y
CONFIG_TMPFS=y
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y

#
# Miscellaneous filesystems
#
CONFIG_ADFS_FS=m
# CONFIG_ADFS_FS_RW is not set
CONFIG_AFFS_FS=m
CONFIG_ASFS_FS=m
# CONFIG_ASFS_RW is not set
CONFIG_HFS_FS=m
CONFIG_HFSPLUS_FS=m
CONFIG_BEFS_FS=m
# CONFIG_BEFS_DEBUG is not set
CONFIG_BFS_FS=m
CONFIG_EFS_FS=m
CONFIG_JFFS_FS=m
CONFIG_JFFS_FS_VERBOSE=0
CONFIG_JFFS_PROC_FS=y
CONFIG_JFFS2_FS=m
CONFIG_JFFS2_FS_DEBUG=0
# CONFIG_JFFS2_FS_NAND is not set
# CONFIG_JFFS2_COMPRESSION_OPTIONS is not set
CONFIG_JFFS2_ZLIB=y
CONFIG_JFFS2_RTIME=y
# CONFIG_JFFS2_RUBIN is not set
CONFIG_CRAMFS=y
CONFIG_VXFS_FS=m
CONFIG_HPFS_FS=m
CONFIG_QNX4FS_FS=m
# CONFIG_QNX4FS_RW is not set
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
# CONFIG_UFS_FS_WRITE is not set

#
# Network File Systems
#
CONFIG_NFS_FS=m
CONFIG_NFS_V3=y
CONFIG_NFS_V4=y
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=m
CONFIG_NFSD_V3=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
CONFIG_LOCKD=m
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=m
CONFIG_SUNRPC=m
CONFIG_SUNRPC_GSS=m
CONFIG_RPCSEC_GSS_KRB5=m
CONFIG_SMB_FS=m
CONFIG_SMB_NLS_DEFAULT=y
CONFIG_SMB_NLS_REMOTE="cp437"
CONFIG_CIFS=m
# CONFIG_CIFS_STATS is not set
# CONFIG_CIFS_XATTR is not set
# CONFIG_CIFS_POSIX is not set
CONFIG_NCP_FS=m
CONFIG_NCPFS_PACKET_SIGNING=y
CONFIG_NCPFS_IOCTL_LOCKING=y
CONFIG_NCPFS_STRONG=y
CONFIG_NCPFS_NFS_NS=y
CONFIG_NCPFS_OS2_NS=y
CONFIG_NCPFS_SMALLDOS=y
CONFIG_NCPFS_NLS=y
CONFIG_NCPFS_EXTRAS=y
CONFIG_CODA_FS=m
CONFIG_CODA_FS_OLD_API=y
CONFIG_AFS_FS=m
CONFIG_RXRPC=m

#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
CONFIG_ACORN_PARTITION=y
CONFIG_ACORN_PARTITION_CUMANA=y
CONFIG_ACORN_PARTITION_EESOX=y
CONFIG_ACORN_PARTITION_ICS=y
CONFIG_ACORN_PARTITION_ADFS=y
CONFIG_ACORN_PARTITION_POWERTEC=y
CONFIG_ACORN_PARTITION_RISCIX=y
CONFIG_OSF_PARTITION=y
CONFIG_AMIGA_PARTITION=y
CONFIG_ATARI_PARTITION=y
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
CONFIG_MINIX_SUBPARTITION=y
CONFIG_SOLARIS_X86_PARTITION=y
CONFIG_UNIXWARE_DISKLABEL=y
CONFIG_LDM_PARTITION=y
CONFIG_LDM_DEBUG=y
CONFIG_SGI_PARTITION=y
CONFIG_ULTRIX_PARTITION=y
CONFIG_SUN_PARTITION=y
CONFIG_EFI_PARTITION=y

#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=m
CONFIG_NLS_CODEPAGE_737=m
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=m
CONFIG_NLS_CODEPAGE_852=m
CONFIG_NLS_CODEPAGE_855=m
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=m
CONFIG_NLS_CODEPAGE_861=m
CONFIG_NLS_CODEPAGE_862=m
CONFIG_NLS_CODEPAGE_863=m
CONFIG_NLS_CODEPAGE_864=m
CONFIG_NLS_CODEPAGE_865=m
CONFIG_NLS_CODEPAGE_866=m
CONFIG_NLS_CODEPAGE_869=m
CONFIG_NLS_CODEPAGE_936=m
CONFIG_NLS_CODEPAGE_950=m
CONFIG_NLS_CODEPAGE_932=m
CONFIG_NLS_CODEPAGE_949=m
CONFIG_NLS_CODEPAGE_874=m
CONFIG_NLS_ISO8859_8=m
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
CONFIG_NLS_ASCII=m
CONFIG_NLS_ISO8859_1=m
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=m
CONFIG_NLS_KOI8_R=m
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_UTF8=m

#
# Library routines
#
CONFIG_CRC_CCITT=m
CONFIG_CRC32=y
CONFIG_LIBCRC32C=m
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=m

#
# HP Simulator drivers
#
# CONFIG_HP_SIMETH is not set
# CONFIG_HP_SIMSERIAL is not set
# CONFIG_HP_SIMSCSI is not set

#
# Profiling support
#
CONFIG_PROFILING=y
CONFIG_OPROFILE=m

#
# Kernel hacking
#
CONFIG_IA64_GRANULE_16MB=y
# CONFIG_IA64_GRANULE_64MB is not set
CONFIG_DEBUG_KERNEL=y
CONFIG_IA64_PRINT_HAZARDS=y
# CONFIG_DISABLE_VHPT is not set
CONFIG_MAGIC_SYSRQ=y
# CONFIG_DEBUG_SLAB is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_IA64_DEBUG_CMPXCHG is not set
# CONFIG_IA64_DEBUG_IRQ is not set
# CONFIG_DEBUG_INFO is not set
CONFIG_SYSVIPC_COMPAT=y

#
# Security options
#
CONFIG_SECURITY=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_CAPABILITIES=m
CONFIG_SECURITY_ROOTPLUG=m
# CONFIG_SECURITY_SELINUX is not set
#
# Security options
#
CONFIG_SECURITY=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_CAPABILITIES=m
CONFIG_SECURITY_ROOTPLUG=m
# CONFIG_SECURITY_SELINUX is not set

#
# Cryptographic options
#
CONFIG_CRYPTO=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=m
CONFIG_CRYPTO_SHA1=m
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_SHA512=m
CONFIG_CRYPTO_DES=m
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_AES=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_ARC4=m
CONFIG_CRYPTO_KHAZAD=m
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_CRC32C=m
CONFIG_CRYPTO_TEST=m

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

Reply-To: iod00d@hp.com

On Thu, Mar 30, 2006 at 11:25:31AM -0800, Andrew Morton wrote:
> Subject: [Bugme-new] [Bug 6307] New: Bug hwclock in kernel 2.6.16
...
> Most recent kernel where this bug did not occur: Kernel 2.6.8-2-itanium-smp 
> (Defauld in Debian 3.1)

I can't reproduce this with kernel.org 2.6.15 on an HP rx2600.


> Software Environment:Debian 3.1

I'm running Debian "testing" on this box.

> Problem Description: running hwclock --hctosys or ntpdate $ip_addr_ntp_server 
> hangup the kernel.

I tried both commands and both seem to work fine.
At least they didn't hang.
I suggest looking at firmware interaction problems.

I'm upgrading system firmware so I can upgrade to 2.6.16.
(SAL needs to be updated)
If the problem is reproducable on 2.6.16, I'll post again.

hth,
grant

>> Subject: [Bugme-new] [Bug 6307] New: Bug hwclock in kernel 2.6.16
>...
>> Most recent kernel where this bug did not occur: Kernel 2.6.8-2-itanium-smp 
>> (Defauld in Debian 3.1)
>
>I can't reproduce this with kernel.org 2.6.15 on an HP rx2600.

My HP rx2620, HP zx2000 and Intel Tiger are all running 2.6.16-gitlatest and
none of them hang on hwclock or ntpdate either.

The config file listed in the bugzilla has a zillion differences from
arch/ia64/defconfig ... and chance of trying with a kernel based on
this config?

-Tony

Reply-To: dannf@debian.org

On 03/30/06 15:22, Luck, Tony wrote:
> >> Subject: [Bugme-new] [Bug 6307] New: Bug hwclock in kernel 2.6.16
> >...
> >> Most recent kernel where this bug did not occur: Kernel 2.6.8-2-itanium-smp 
> >> (Defauld in Debian 3.1)
> >
> >I can't reproduce this with kernel.org 2.6.15 on an HP rx2600.
> 
> My HP rx2620, HP zx2000 and Intel Tiger are all running 2.6.16-gitlatest and
> none of them hang on hwclock or ntpdate either.
> 
> The config file listed in the bugzilla has a zillion differences from
> arch/ia64/defconfig ... and chance of trying with a kernel based on
> this config?

We've put 3 2.6.16 builds in Debian/sid so far (2.6.16-[234]).
This user didn't specify which one, but I was unable to reproduce on
my HP rx2600 with either -3 or -4, and I see nothing in the -3
changelog that looks like it could've caused this behavior:
  http://changelog.debian.net/linux-2.6

I'm compile kernel 2.6.15 and install unstable debian kernel 2.6.16-3,4, but 
thet not to help.

For experiments be able this server (xSeries 455 Itanium2 x 2 cpu).

I post a new information about this bug:

Unable to handle kernel NULL pointer dereference (address 0000000000000018)
ntpdate[2430]: Oops 8804682956800 [1]
Modules linked in: shpchp pci_hotplug i2c_viapro via686a i2c_isa i2c_core generi
c usbhid uhci_hcd usbcore via82cxxx ide_generic ide_disk ide_cd cdrom ide_core

Pid: 2430, CPU 0, comm:              ntpdate
psr : 0000101008022018 ifs : 8000000000000307 ip  : [<a0000001000a4370>]    Not
tainted
ip is at time_interpolator_reset+0x30/0x3a0
unat: 0000000000000000 pfs : 000000000000048c rsc : 0000000000000003
rnat: 0000000000000000 bsps: 0000000000000000 pr  : 00000000001adad9
ldrs: 0000000000000000 ccv : 0000000000000000 fpsr: 0009804c0270033f
csd : 0000000000000000 ssd : 0000000000000000
b0  : a000000100097120 b6  : a000000100002d70 b7  : a0000001002539e0
f6  : 000000000000000000000 f7  : 000000000000000000000
f8  : 000000000000000000000 f9  : 000000000000000000000
f10 : 000000000000000000000 f11 : 000000000000000000000
r1  : a000000100994f30 r2  : 0000000000000000 r3  : 00000000fffffeff
r8  : 0000000000000018 r9  : 0000000000000000 r10 : a000000100794f60
r11 : 0000000000000040 r12 : e0000001ba9dfe00 r13 : e0000001ba9d8000
r14 : 0000000000fa0000 r15 : a0000001007a7938 r16 : a0000001007a6980
r17 : a0000001007a6988 r18 : a0000001007a6960 r19 : a0000001007a7938
r20 : a0000001007a6980 r21 : a0000001007a6988 r22 : 0000000000000000
r23 : 0000000004b60e00 r24 : 0000000018148d00 r25 : ffffffffbb8e4612
r26 : 0000000000002971 r27 : a0000001007e27b8 r28 : a0000001007e27a8
r29 : 000000004471bbba r30 : a0000001007e27a0 r31 : a0000001007e27b0

Call Trace:
 [<a000000100012620>] show_stack+0x80/0xa0
                                sp=e0000001ba9df980 bsp=e0000001ba9d9258
 [<a000000100012ea0>] show_regs+0x800/0x820
                                sp=e0000001ba9dfb50 bsp=e0000001ba9d91f0
 [<a00000010003a4d0>] die+0x210/0x320
                                sp=e0000001ba9dfb60 bsp=e0000001ba9d91a8
 [<a00000010005aca0>] ia64_do_page_fault+0x1c0/0x980
                                sp=e0000001ba9dfb80 bsp=e0000001ba9d9140
 [<a00000010000c760>] ia64_leave_kernel+0x0/0x290
                                sp=e0000001ba9dfc30 bsp=e0000001ba9d9140
 [<a0000001000a4370>] time_interpolator_reset+0x30/0x3a0
                                sp=e0000001ba9dfe00 bsp=e0000001ba9d9108
 [<a000000100097120>] do_settimeofday+0x1c0/0x240
                                sp=e0000001ba9dfe00 bsp=e0000001ba9d90c0
 [<a000000100095890>] do_sys_settimeofday+0x170/0x2a0
                                sp=e0000001ba9dfe00 bsp=e0000001ba9d9078
 [<a000000100095b50>] sys_settimeofday+0x190/0x1e0
                                sp=e0000001ba9dfe00 bsp=e0000001ba9d9010
 [<a00000010000c5c0>] ia64_ret_from_syscall+0x0/0x20
                                sp=e0000001ba9dfe30 bsp=e0000001ba9d9010

This bug is a little stale.

Sergey: Any update? Does this issue still exist w/ 2.6.17 or 2.6.18-rc1?

Chris: Mind taking a peek at this and seeing if someone could try to reproduce
it on an x455?

This bug is quite stale. If it cannot be reproduced soon I'm going to close this.

I am getting a HWCLOCK failure with latest and updated FC6 installed on HP
Pavillion a1510y

I first noticed it in bootup and shutdown logs.  Then manually invoking hwclock
it is unable to read the clock.

What information do you need?

Here is error message


# hwclock --show
select() to /dev/rtc to wait for clock tick timed out

I have the same issue on my HP DV6110 running Fedora Core 6.  2.6.20-1.2933.fc6
and 2.6.20-1.2944.fc6  Let me know what other information you need.  The bug
seems to occur when the difference between the system clock and the hw clock is
high.  I typically had the system  hang on start up /  shut down occasssionaly
but when I was travelling i switched from EDT to PDT, then switch back to EDT
and now it happens every time I run hwclock --systohc.

2.6.24-rc3 includes a fix for the NTP code, related to hwclock and CMOS writes. Could you check whether this issue still occurs with that (or newer) kernels?

I have the same problem on a HP dv6155eu laptop. I also tested it with vanilla-sources-2.6.24_rc3.
The system freezes when I do a "hwclock --show" with or without --directisa.

So far I have tested this with the following kernel versions:
The freeze occurs in the following versions:
2.6.24_rc3 64bit (vanilla)
2.6.23 64bit (+tuxonice,notick,ck-patches,...)
2.6.22 32bit+64bit (kubuntu-7.10)
2.6.20 32bit (grml-1.0)
2.6.20 32bit (kubuntu-7.04)
2.6.20.11 64bit (grml64-0.1)

For my oldest Live-CD it seems to work:
2.6.16 32bit (grml-0.7)

I hope to be able to confirm that as soon as I'm able to compile a vanilla 2.6.16 kernel, and then work upwards until I find the first version that doesn't work.

> I hope to be able to confirm that as soon as I'm able to compile a 
> vanilla 2.6.16 kernel, and then work upwards until I find the first 
> version that doesn't work.

once you are able to build kernels manually, you might want to try 
git-bisect - that should pinpoint the exact commit (out of 60,000 
commits) that broke this - with 10-20 kernel rebuilds (worst-case). If 
vanilla 2.6.16 indeed works then you can set the bisection up like this:

1)

install the 'git' package. (if you dont have it already.)

2)

clone Linus' latest tree. (if you dont have it already):

 git-clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6.git

this will take some time to complete, but fetching the latest Linux 
source will be a lot faster as all subsequent pulls will be deltas. You 
can do it via:

 cd linux-2.6.git
 git-pull

that will fetch Linus'-tree-of-the-minute. It does not get any more 
bleeding edge than that. This is a fully functional Linux kernel tree 
which you can build just fine. All the git stuff is under 
linux-2.6.git/.git. If you patch your kernel and want to restore to the 
pristine git code just do:

 cd linux-2.6.git
 rm -rf *
 git-checkout .

3)

once you have the git tree, start bisection via:

 git-bisect start

build the kernel and check that it is a 'bad' kernel, and tell the 
bisector that it's a bad kernel:

 git-bisect bad

Tell the bisection engine that v2.6.16 is a "good" kernel:

 git-bisect good v2.6.16

at this point git-bisect will calculate a 'middle' commit and switches 
to it and checks the files out - build that kernel and test it. If it's 
"good", mark it such:

 git-bisect good

if it's "bad", tell the bisection engine about it:

 git-bisect bad

the number of commits will halve at every step - and eventually 
git-bisect will give you a single commit that introduces the breakage. 
Hopefully that commit will be something related to the RTC :-)

I did use git-bisect as described above with the following result:
# git-bisect bad
b7408aff2d325581dcafffa5dbcc09c42ae64b5d is first bad commit
commit b7408aff2d325581dcafffa5dbcc09c42ae64b5d
Author: Richard Purdie <rpurdie@rpsys.net>
Date:   Mon Jun 19 13:08:39 2006 +0100

    [ARM] 3563/1: LED: Set the LOCOMO LED driver default triggers

    Patch from Richard Purdie

    Set the default triggers for the LOCOMO LED driver.

    Signed-off-by: Richard Purdie <rpurdie@rpsys.net>
    Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

:040000 040000 963af85a141b81ab2abef1654e1cb596d7d2918c cf41f7b8cb1378131f70eaadb1819dea27a8667a M      drivers

# git-bisect log
git-bisect start
# bad: [e1cca7e8d484390169777b423a7fe46c7021fec1] Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/hpa/linux-2.6-x86setup
git-bisect bad e1cca7e8d484390169777b423a7fe46c7021fec1
# good: [7705a8792b0fc82fd7d4dd923724606bbfd9fb20] Linux 2.6.16
git-bisect good 7705a8792b0fc82fd7d4dd923724606bbfd9fb20
# bad: [5f0b1437e0708772b6fecae5900c01c3b5f9b512] Merge master.kernel.org:/pub/scm/linux/kernel/git/jejb/scsi-misc-2.6
git-bisect bad 5f0b1437e0708772b6fecae5900c01c3b5f9b512
# bad: [eb2971b68a7d17a7d0fa2c7fc6fbc4bfe41cd694] [XFRM] STATE: Search by address using source address list.
git-bisect bad eb2971b68a7d17a7d0fa2c7fc6fbc4bfe41cd694
# good: [98174e46974323e4941c72e46345f7277755e146] Merge HEAD from ../linux-2.6
git-bisect good 98174e46974323e4941c72e46345f7277755e146
# bad: [47c2a3aa4475d27073dd3c7e183fcc13f495c8f5] genirq: add chip->eoi(), fastack -> fasteoi
git-bisect bad 47c2a3aa4475d27073dd3c7e183fcc13f495c8f5
# bad: [6edad161cd4dfe1df772e7a74ab63cab53b5e8c1] Merge branch 'upstream-linus' of master.kernel.org:/pub/scm/linux/kernel/git/jgarzik/libata-dev
git-bisect bad 6edad161cd4dfe1df772e7a74ab63cab53b5e8c1
# bad: [d588fcbe5a7ba8bba2cebf7799ab2d573717a806] Merge master.kernel.org:/pub/scm/linux/kernel/git/gregkh/i2c-2.6
git-bisect bad d588fcbe5a7ba8bba2cebf7799ab2d573717a806
# bad: [22ae813b85df7c0b0fc7c8d6f336d6a9f566ff97] add __iowrite64_copy
git-bisect bad 22ae813b85df7c0b0fc7c8d6f336d6a9f566ff97
# good: [be967b7e2f7747a5ebf2a07ee627d9338491e784] Merge git://git.infradead.org/mtd-2.6
git-bisect good be967b7e2f7747a5ebf2a07ee627d9338491e784
# good: [d9eaec9e295a84a80b663996d0489fcff3a1dca9] Merge branch 'audit.b21' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/audit-current
git-bisect good d9eaec9e295a84a80b663996d0489fcff3a1dca9
# bad: [905f14672e6d0552bfde954d5f7adb5f2c7a7960] [ARM] Fix tosa build error
git-bisect bad 905f14672e6d0552bfde954d5f7adb5f2c7a7960
# bad: [55c20c0af7fe7d5d09af4addfafcfe3bdc500f5d] [ARM] 3599/1: AT91RM9200 remove global variables
git-bisect bad 55c20c0af7fe7d5d09af4addfafcfe3bdc500f5d
# bad: [7238d7ee82d325212e83630047e9844943225118] [ARM] 3586/1: AT91RM9200 header update
git-bisect bad 7238d7ee82d325212e83630047e9844943225118
# bad: [10e8e1fb758eed5cfb0cae1b770f842624851e7b] [ARM] 3581/1: AT91RM9200 Internal SRAM
git-bisect bad 10e8e1fb758eed5cfb0cae1b770f842624851e7b
# bad: [91f8ed835ffb34b4108cc16eefd3303e4068bee0] [ARM] 3578/1: AT91RM9200 Clock update
git-bisect bad 91f8ed835ffb34b4108cc16eefd3303e4068bee0
# bad: [b7408aff2d325581dcafffa5dbcc09c42ae64b5d] [ARM] 3563/1: LED: Set the LOCOMO LED driver default triggers
git-bisect bad b7408aff2d325581dcafffa5dbcc09c42ae64b5d

As the freeze does not occur on every call to hwclock I made a script that called it 200 times in a row. All revisions marked with good passed this test. All revisions marked as bad froze after 35 iterations maximum (most times it was below 5 times). I tested those failing after more than 25 iterations again and they failed earlier the next time.

I hope this helps.

> I did use git-bisect as described above with the following result:
> # git-bisect bad
> b7408aff2d325581dcafffa5dbcc09c42ae64b5d is first bad commit
> commit b7408aff2d325581dcafffa5dbcc09c42ae64b5d
> Author: Richard Purdie <rpurdie@rpsys.net>
> Date:   Mon Jun 19 13:08:39 2006 +0100
> 
>     [ARM] 3563/1: LED: Set the LOCOMO LED driver default triggers
> 
>     Patch from Richard Purdie

does the lockup go away if you unapply the bad commit from the freshest 
tree? (i.e. from 'master', the 2.6.24-rc3-git6-ish kernel?)

you can extract the diff of the bad commit via:

  git-log -1 -p b7408aff2d32558 > bad.patch

and then do:

  patch -p1 -R < bad.patch

(-R is the 'revert' option.)

to apply the patch again, do:

 patch -p1 < bad.patch

It does not work when I unapply this commit. I didn't expect it to, since I don't use this LED driver or even have it enabled.
What should be the next step? Go back to the highest "good" revision and test it further so to confirm the result?

I just tried the last good version again and played around with git-diff a bit:

When I understand this the right way:
git-diff d9eaec9e295a84a80b663996d0489fcff3a1dca9 b7408aff2d325581dcafffa5dbcc09c42ae64b5d
should be the same diff that I get when I use git-log -1 -p b7408aff2d32558, as d9.. is the last working version and b74... is the first failing one. But the diff between the two is a huge thing not just the small patch that I get from git-log.

i've replayed your bisection log and it hones in on that LEDS commit, so 
believe one of the good/bad decisions was wrong. (or, the bug could be 
"spurious" or nondeterministic in some other way - but lets exclude that 
possibility for now because you managed to find a way to reproduce the 
lockup so well.)

to not have to redo the whole bisection, i'd suggest to replay up until 
the last "good" kernel, just paste these commands into a shell prompt:

 git-bisect start
 git-bisect bad e1cca7e8d484390169777b423a7fe46c7021fec1
 git-bisect good 7705a8792b0fc82fd7d4dd923724606bbfd9fb20
 git-bisect bad 5f0b1437e0708772b6fecae5900c01c3b5f9b512
 git-bisect bad eb2971b68a7d17a7d0fa2c7fc6fbc4bfe41cd694
 git-bisect good 98174e46974323e4941c72e46345f7277755e146
 git-bisect bad 47c2a3aa4475d27073dd3c7e183fcc13f495c8f5
 git-bisect bad 6edad161cd4dfe1df772e7a74ab63cab53b5e8c1
 git-bisect bad d588fcbe5a7ba8bba2cebf7799ab2d573717a806
 git-bisect bad 22ae813b85df7c0b0fc7c8d6f336d6a9f566ff97
 git-bisect good be967b7e2f7747a5ebf2a07ee627d9338491e784

NOTE: at this point it will suggest the d9eaec9e29 commit, but please 
double-check it whether it's indeed a "good" kernel. Continue from that 
point on and double-check every bisection point. Either this last 'good' 
decision, or one of the following 'bad' decisions could be wrong:

 905f14672e6d0552bfde954d5f7adb5f2c7a7960
 55c20c0af7fe7d5d09af4addfafcfe3bdc500f5d
 7238d7ee82d325212e83630047e9844943225118
 10e8e1fb758eed5cfb0cae1b770f842624851e7b
 91f8ed835ffb34b4108cc16eefd3303e4068bee0
 b7408aff2d325581dcafffa5dbcc09c42ae64b5d

it is not typical for a bisection to end with a series of 7 
'bad' bisection points - the chance for that is roughly 1/128. So 
somewhere around commits d9eaec9e295, 905f14672e or 55c20c0af7 you might 
have made a mistake.

If you still get the same result then we'll have to rethink the whole 
strategy - but i think the fact that it's 7 'bad' bisection points at 
the end strongly suggests an incorrect bisection point at one of the 
aforementioned points.

and this happens to me quite frequently too - even though i'm quite 
careful and experienced at it, roughly 1 out of 5 of my own bisections 
fails due to a bad decision. That's not a surprise: with 15 bisection 
points each, even a seemingly 5% failure/mistake rate multiplies up very 
quickly, to 1.05^15 == 2.05, a 50% total failure rate!

so i have automated certain types of bisections around the "git-bisect 
run" feature, but such automation cannot be universally applied to hard 
lockup bugs like yours.

This time I wrote a more detailed log:

git-bisect start
# good: [d9eaec9e295a84a80b663996d0489fcff3a1dca9] Merge branch 'audit.b21' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/audit-current
git-bisect good d9eaec9e295a84a80b663996d0489fcff3a1dca9 # 200+
# bad: [92d499d991ec4f5cbd00d6f33967eab9d3ee8d6c] Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband
git-bisect bad 92d499d991ec4f5cbd00d6f33967eab9d3ee8d6c  # 0 | 0
# good: [e5b9ddd9a0f95e133db7b43d05978f24cd6f1369] skge: turn carrier off when down
git-bisect good e5b9ddd9a0f95e133db7b43d05978f24cd6f1369 # 100 | 100
# bad: [7742c0bc8594cf1bcbad1f3e876acae5b644dc7b] Merge branch 'upstream-linus' of master.kernel.org:/pub/scm/linux/kernel/git/jgarzik/netdev-2.6
git-bisect bad 7742c0bc8594cf1bcbad1f3e876acae5b644dc7b  # 0 | 0
# bad: [22adb358e816ce6aa0afb231ae9d826b0bddc8b0] [SPARC64]: Eliminate NR_CPUS limitations.
git-bisect bad 22adb358e816ce6aa0afb231ae9d826b0bddc8b0  # 0 | 0
# bad: [6de410c2b0cc055ae9ee640c84331f6a70878d9b] Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/avi/kvm
git-bisect bad 6de410c2b0cc055ae9ee640c84331f6a70878d9b  # 4 | 4
# good: [f73b0a08eae0e28c50db5dd5ab8245546918bfb6] Merge branch 'upstream-linus' of master.kernel.org:/pub/scm/linux/kernel/git/jgarzik/libata-dev
git-bisect good f73b0a08eae0e28c50db5dd5ab8245546918bfb6 # 100 | 100
# bad: [fa0a00910925fdbd3a528404a47817b3a9fda5be] x86-64: Drop -traditional for arch/x86_64/boot
git-bisect bad fa0a00910925fdbd3a528404a47817b3a9fda5be  # 8 | 2
# good: [24a77daf3d80bddcece044e6dc3675e427eef3f3] Merge branch 'for-2.6.22' of git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc
git-bisect good 24a77daf3d80bddcece044e6dc3675e427eef3f3 # 100 | 100
# bad: [19d1743315099665db4ce02c9942507a5ee1deea] i386: Simplify smp_call_function*() by using common implementation
git-bisect bad 19d1743315099665db4ce02c9942507a5ee1deea  # 2 | 1
# bad: [d6454706c382ab74e2ecad7803c434cc6bd30343] Merge branch 'for-linus' of master.kernel.org:/pub/scm/linux/kernel/git/jikos/hid
git-bisect bad d6454706c382ab74e2ecad7803c434cc6bd30343  # -1 | 100 | 100 + froze at once with --directisa; this kernel showed a remakable delay before the login prompt was presented (>20s after the last init script finished)

!! I modified the test-script: it now calls hwclock 50 times without --directisa and 50 times with it
# good: [152a6a9da1bd3ed5dcbbf6ff17c7ebde0eb9a754] Merge master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
git-bisect good 152a6a9da1bd3ed5dcbbf6ff17c7ebde0eb9a754 # 100 | 100 - this one had a similar delay
# good: [713c8aad6b7202671ce1ac6109f6b48d8223e938] USB HID: numlock quirk for dell W7658 keyboard
git-bisect good 713c8aad6b7202671ce1ac6109f6b48d8223e938 # 100 | 100
# good: [8222fbe67cc6c83bb6d8d9d913aee17957fc0654] USB HID: clarify static quirk handling as squirks
git-bisect good 8222fbe67cc6c83bb6d8d9d913aee17957fc0654 # 100 | 100
# good: [f61c9127b9840661244b1b6571e4304a7f0b5c73] USB HID: don't warn on idVendor == 0
git-bisect good f61c9127b9840661244b1b6571e4304a7f0b5c73 # 100 | 100
# good: [11941a321d49cd2cafc8e64f66cbfed60fc1c691] Merge branch 'field-zeroing' into for-linus
git-bisect good 11941a321d49cd2cafc8e64f66cbfed60fc1c691 # 100 | 100

d6454706c382ab74e2ecad7803c434cc6bd30343 is first bad commit

"|" marks a reboot.
The numbers show how often "hwclock --show" was called successfully before the system froze
- 100 or above means that there was no hangup at all.
- -1 means that not even "setting system clock to hardware clock" worked

The method to extract a patch with git-log -p -1 d6454706c382ab74e2ecad7803c434cc6bd30343 didn't work for me this time, so I was unable to check if unapplying it from HEAD would help

Sorry for the double post.
I forgot to say that I started with good: [d9eaec9e295a84a80b663996d0489fcff3a1dca9] as it was the best tested version at this time.

btw., could you try to describe in what way the system freezes? Is it a hard hang - numlock led does not work, etc. - or a soft hang?

The system locks up completely: That means processor fan runs at full speed (after a few seconds) and there is no reaction to any user input, not even the LEDs for caps lock or num lock work.

d6454706c3 is a "merge commit" - an artificial commit that shows that someone (Linus, most of the time) has done manual merging. A merge commit is only metadata, it does not change the kernel source.

So somewhere along the line some commit was probably misidentified. Perhaps the method of identifying "good" kernels is inadequate?

Basically, every "bad" bisection point is 100% reliable (as long as you have not typoed it) - because if a kernel hung up even just one that means it's 100% bad. If a kernel passed 100 iterations that does not mean it's necessarily "good". (but it is a strong indicator)

I'm looking at the following two points right now:

 git-bisect good 24a77daf3d80bddcece044e6dc3675e427eef3f3
 git-bisect bad 19d1743315099665db4ce02c9942507a5ee1deea

and they _do_ seem plausible and narrow things down quite a bit. You can check the "suspected" commits via: "git-bisect visualize". This command will show you all the candidate commits that are still remaining between the last "good" and "bad" bisection points. There are a good deal of time code and architecture code related commits in that set of commits.

Could you check one of those bisection points (dc87c3985) manually, via:

 git-bisect start
 git-bisect good 24a77daf3d80bddcece044e6dc3675e427eef3f3
 git-bisect bad 19d1743315099665db4ce02c9942507a5ee1deea
 git-reset --hard dc87c3985e9b442c60994308a96f887579addc39
 git-checkout .

this gives you that commit, and after you have tested it you can do "git-bisect good" or "git-bisect bad" and you can continue with the bisection.

i picked this dc87c3985e commit from the visualized graph: it is one commit earlier than the stream of those "suspect" commits. So my guess would be that it's still a "good" kernel. One of the commits in the dc87c3985e..19d17433150 range could be the guilty commit.

Also, could you attach to this bugzilla the output file of the following script:

  http://people.redhat.com/mingo/cfs-scheduler/tools/cfs-debug-info.sh

best run it on a known "bad" kernel. (before the lockup) This script outputs a laundry list of system details so we'll have a good overview of what hardware environment this is.

also, could you attach the .config as well that you are using. (of a "bad" kernel, preferably.)

> The system locks up completely: That means processor fan runs at full 
> speed (after a few seconds) and there is no reaction to any user 
> input, not even the LEDs for caps lock or num lock work.

ok, that's good. Does the NMI watchdog work on your system? Add the 
nmi_watchdog=2 [or perhaps nmi_watchdog=1, if you system has an IO-APIC] 
boot option to the 'kernel' line of /etc/grub.conf. This feature will 
print a useful stack backtrace of hard lockups after 10-20 seconds, as 
long as you are on a text console. (or via serial if you have a serial 
console configured and attached via a null modem cable to another 
system.)

first make sure it works fine: boot into a known-bad kernel (preferably 
2.6.24-rcx) with nmi_watchdog=2 kernel boot option and check that the 
NMI count(s) in /proc/interrupts is increasing periodically. If they do 
it's OK and any hard lockup will hopefully be followed up by a useful 
stack dump printed by the kernel. Write the EIP line and the "Call 
Trace" lines down or make a digital picture of it.

if it does not work then you'll be met with a sad:

  NMI:          0          0   Non-maskable interrupts

output, or sometimes the NMI counts are stuck at a low count:

  NMI:       1200       1230   Non-maskable interrupts

and dont increase. In this case the hard lockup wont be detected by the 
NMI.

Created attachment 13829 [details]
cfs-debug-info-2007.12.02-19.05.12

Created attachment 13830 [details]
config-2.6.24_rc3

I attached the debug information and the kernel config - both for the 2.6.24_rc3 kernel.

I didn't get the NMI Watchdog thing to work, but it still may be a configuration issue. I just activated the watchdog software driver.
with nmi_watchdog=1 it gets stuck at low numbers; with 2 it's at 0
I hope you can help me with this.

I tried the revision you proposed and it showed that the freeze is less reproducible than I thought: First it got stuck at the first test and then it passed 100 two times in a row. So I guess there is no point in bisecting anymore.

> I tried the revision you proposed and it showed that the freeze is 
> less reproducible than I thought: First it got stuck at the first test 
> and then it passed 100 two times in a row. So I guess there is no 
> point in bisecting anymore.

i think your bisection results so far are excellent and have narrowed 
down the bug quite a bit. Bisection is still a powerful tool even if a 
hang is nondeterministic to reproduce, because we can definitely 
identify 'bad' commits.

could you try to run a really long test for commit 24a77daf3d80bd, so 
that we establish that it's a 100% "good" kernel? That way we know that 
it's one of the 36 commits in that range. Between 2.6.16 and 2.6.24-rc3 
there are 53317 commits (!), so we've come quite a long way. Those 36 
commits definitely look "suspect" in the sense that they change 
hardware-relevant code on x86.

i found this in the boot log:

| Nvidia board detected. Ignoring ACPI timer override.
| If you got timer trouble try acpi_use_timer_override

not that it should make much of a difference, but might be worth trying: 
add acpi_use_timer_override to the boot options.

> I didn't get the NMI Watchdog thing to work, but it still may be a 
> configuration issue. I just activated the watchdog software driver. 
> with nmi_watchdog=1 it gets stuck at low numbers; with 2 it's at 0 I 
> hope you can help me with this.

for some reason the NMIs are not coming - this is not your fault.

does it get any better if you boot with the idle=poll option? Sometimes 
this makes nmi_watchdog=2 work better. (because the CPU does not enter 
C2 mode)

I made a script that randomly calls: "hwclock --(show|systohc|hctosys)( --directisa)?"
This makes 24a77daf3d80bd freeze after less than 100 iterations.
So it seems that it is not a good kernel after all.

I'm now running the same script with the 2.6.16 based LiveCD (grml-0.7 i386). It is now at ~700 iterations.

Btw. the earliest kernel I am able to run from the git-repository is around the 2.6.17 release. Earlier kernels don't seem to include the nvidia SATA driver that I need to access the hard drive. At least not for x86_64.

The idle=poll thing did indeed fix the issue with NMI so I will try to get the first "screenshot".

With acpi_use_timer_override it is now at 150 on 24a77daf3d80bd. I'm going to try it now with 2.6.24_rc3.

2.6.24_rc3 still hangs after a few iterations, but NMI seems to have no effect.
The counter in /proc/interrupts increases, but I'm not getting any debug information when the system freezes.

Created attachment 13834 [details]
simulate a hard lockup

Could you try the attached lockupcli.c code, does the NMI watchdog print anything, or does it lock up silently?

You have to be on a VGA text console to see NMI watchdog output normally. (run lockupcli from there too)

normally you should get a 'NMI watchdog lockup detected' type of message within a minute after experiencing the hard lockup.

If you get it in lockupcli, but the hwclock lockup does _not_ get detected by the NMI watchdog, then this is possibly some deep, hardware level lockup (locked up system bus or something) so that there is no real CPU activity anymore. (and no NMIs)

In that case the only real debug tool would be bisection - which will be pretty hard if the last testable 2.6.17-ish git kernel still locks up for you ...

I have never used the VGA text console only the framebuffer console, so this would be the reason why I don't get any watchdog output.
Could you please point me to a relevant documentation about how to get to a VGA console?

> I have never used the VGA text console only the framebuffer console, 
> so this would be the reason why I don't get any watchdog output. Could 
> you please point me to a relevant documentation about how to get to a 
> VGA console?

hm, i _think_ the framebuffer console should be able to be printed to 
from the NMI watchdog as well - but generally it's a bit more fragile.  
Do you get NMI watchdog output from lockupcli? If not then the problem 
is likely the fbcon that is preventing it from being printed.

I started with vga=ask now and chose a VGA mode. Now I'm getting output from watchdog when I run lockupcli. The problem is: I'm still not getting anything after a hwclock freeze.

> I started with vga=ask now and chose a VGA mode. Now I'm getting 
> output from watchdog when I run lockupcli. The problem is: I'm still 
> not getting anything after a hwclock freeze.

ok, this suggests some "harder than hard lockup" wedge of the system :-/

does that 2.6.17-ish kernel (which has the required SATA driver you 
need) lock up too? If yes then there's no good way to bisect this i 
suspect. If yes then bisection might be a workable (albeit very hard) 
option still.

if you are on a vga console, then could you try to run "strace hwclock 
--show" in a loop, and see which is the last line you get before it 
locks up? It will likely lock up in an ioctl(4) [that is where hwclock 
calls the kernel's RTC driver] - could you write down that exact ioctl 
line? (and attach the output of "strace -o strace.log hwlock --show" of 
a non-locked-up instance of hwclock, for comparison)

It's time to sleep for today so I will continue tomorrow. I will provide you with the strace output.

Another thought is: The LiveCD has a 2.6.16 i386 kernel obviously _with_ the neede d driver, and it's working there (hopefully 700 iterations with my shiny new hwclock torture script are enough). So in the worst case I could set up a 32bit system and continue bisecting there. How can I find out with which revision nv_sata was added for x86/x86_64? 

> Another thought is: The LiveCD has a 2.6.16 i386 kernel obviously 
> _with_ the neede d driver, and it's working there (hopefully 700 
> iterations with my shiny new hwclock torture script are enough). So in 
> the worst case I could set up a 32bit system and continue bisecting 
> there. How can I find out with which revision nv_sata was added for 
> x86/x86_64?

hm, in theory such drivers are available to all platforms at once. And 
often "support" means the introduction of relatively small amount of 
code, so i couldnt tell it right away. Vanilla 2.6.16 x86_64 definitely 
didnt work for you?

"git-log --follow drivers/ata/sata_nv.c" shows a really long history 
going back to 2.6.12-rc2, when git history started. Perhaps what broke 
x86_64 for you was some subtle .config change that resulted in the 
disabling of the driver?

Created attachment 13836 [details]
strace-hwclock.log

I attached the strace output for a normal run of hwclock --show.

When it freezes the last lines look like this:
ioctl(3, RTC_RD_TIME, {tm_sec=2, tm_min=59, tm_hour=12, tm_mday=1, tm_mon=11, tm_year=107, ...}) = 0
... a screen full of identical ioctl calls ....
ioctl(3, RTC_RD_TIME, {tm_sec=2, tm_min=59, tm_hour=12, tm_mday=1, tm_mon=11, tm_year=107, ...}) = 0
ioctl(3, RTC_RD_TIME

The last line is cut after RTC_RD_TIME and before the comma.

With early kernels the problem was that the option in menuconfig (and make oldconfig was unable to assign the symbol)  simply vanished (other drivers in the same category are still there). I'm going to try to configure a 2.6.16 kernel again today.

> The last line is cut after RTC_RD_TIME and before the comma.

ok, that narrows down the lockup site quite a bit - the likely source is 
the rtc_get_rtc_time() function.

if you do:

  git-blame drivers/char/rtc.c

and search for rtc_get_rtc_time then you can see how various commits 
affected this particular function.

much of this function stayed unchanged since 2.6.12, which eases 
analysis. So excluding the 1da177e commit, the following commits 
affected this function directly:

  0f749646
  358333a0
  403fe5ae
  47f176fd
  b7599587

i got this list of commits via:

   grep -v 1da177e rtc_get_rtc_time.c | cut -d' ' -f1 | sort  | uniq

( where rtc_get_rtc_time.c is a new file i created by pasting the 
  rtc_get_rtc_time portion of the git-blame output. Someone should 
  extend git-log/git-blame to filter down to the function level. )

here's an analysis of those commits:

- commit 358333a0 is a pretty harmless cleanup of commit 403fe5ae.

- commit 0f749646 i'd exclude as a candidate bad patch. (primarily 
  because i wrote it and i always write perfect code, secondarily 
  because this cannot affect an ioctl level call.)

- commit 47f176fd looks harmless too. (not the least because it was
  Acked-by me. It introduced also a bug that 403fe5ae then fixes. I did
  not ack that bug, of course.)

- commit 403fe5ae could be bad in theory - it changes an msleep() to
  a jiffies based loop.

- commit b7599587 is something that could be suspect too: it reads a new
  CMOS register that was never read before. This got introduced in the
  2.6.16 timeframe, so it roghly lines up with your timeline of the 
  breakage.

so i think we've narrowed it down to two commits that changed this 
function. (and of course any indirect change could affect this function 
as well, so these 2 commits are not necessarily a given as the source of 
your regression)

to undo commit b7599587 (for testing purposes) on a recent kernel i'd 
suggest you edit drivers/char/rtc.c and change the following line:

   rtc_tm->tm_wday = CMOS_READ(RTC_DAY_OF_WEEK);

to:

   rtc_tm->tm_wday = 0;

this avoids the new CMOS read quite efficiently.

to undo commit 403fe5ae (for testing purposes) i'd suggest you just 
change this code:

        while (rtc_is_updating() != 0 && jiffies - uip_watchdog < 2*HZ/100)
                cpu_relax();

to:

        while (rtc_is_updating() != 0 && jiffies - uip_watchdog < 2*HZ/100)
                mdelay(20);

perhaps a too frequent polling of the RTC_FREQ_SELECT register of your 
CMOS wedges the hardware? If i were forced to pick one commit only i'd 
guess this latter one as the source of the regression.

I forgot to say that the log was generated on 2.6.24_rc3.

I got the 2.6.16 kernel to compile, but I had to use a makefile patch to get rid of:
ld: i386:x86-64 architecture of input file `arch/x86_64/boot/compressed/head.o' is incompatible with i386 output

I'm now beginning to test 2.6.16 and when it's "good" I will start bisecting again taking 2x1000 iterations of the new test script separated by a reboot as good.

When I run the tests on 2.6.16 I observed another thing by chance:
The test itself runs perfectly (not a single freeze) but switching to another console via (strg+alt+f2) almost immediately produces a freeze, after the new console is displayed.
And now yet another: After stopping the tests I used ntpdate to get the right time again (somehow the tests lead to ~2 days getting lost ~166000s offset). After that doing a hwclock --systohc leads to a lock up.

This is all with v2.6.16. Both things worked fine when I tried to reproduce this with the LiveCD.
I didn't catch this things before because they wouldn't show up if I didn't use --systohc.

I'm sorry to say, none of your suggested modifications did fix 2.6.24_rc3 :(

Oh sorry that should say _didn't_ fix not did.

So now I'm going mad too. "Did" was alright after all - I just forgot to read the "none". -> "It's still broken" to say it in another way.

> I'm sorry to say, none of your suggested modifications did fix 
> 2.6.24_rc3 :(

hm. Does it get any better if you replace the rtc_is_updating() line in 
rtc_get_rtc_time() with a brute-force mdelay(100) line?

also, can you still reproduce the lockup if you use only a single core, 
i.e. if you boot with maxcpus=1 ?

To replace rtc_get_rtc_time() did not make a difference, but setting maxcpus=1 after that did. It now seems to run fine, even with the extended version of my testscript and even when switching to another terminal. The next thing I try is to undo the changes to rtc.c and then try again with maxcpus=1 I guess.

I can confirm now that 2.6.24_rc3 works well with maxcpus=1.

> I can confirm now that 2.6.24_rc3 works well with maxcpus=1.

ok, that's quite a clue. Could you try the following experiment: boot 
with maxcpus=2 [i.e. both cores active], but restrict the _test-script_ 
itself to CPU#0, via binding the shell from which you start it to CPU#0:

   taskset -p 01 $$

all commands you start after this will be bound to CPU#0.

(the RTC/CMOS might still be accessed from another CPU, by the timer 
interrupt for example - but perhaps the key is to do all of the hwclock 
CMOS accesses from CPU#0.)

another thing, could you enable the following options in your .config 
(on 2.6.24-rc-latest):

 CONFIG_DEBUG_KERNEL=y
 CONFIG_PROVE_LOCKING=y

and see whether you get any extra debug messages before the lockup 
happens?

taskset does not help with maxcpus=2 -> It froze at the first call of hwclock --show

I get nothing else when I enable the debugging options you described. At least not  in a framebuffer console.

> I get nothing else when I enable the debugging options you described. 
> At least not in a framebuffer console.

ok, just wanted to make sure we dont miss something obvious. 
(framebuffer console is fine as long as you made at least one successful 
hwclock command invocation before getting any lockup.)

> taskset does not help with maxcpus=2 -> It froze at the first call of 
> hwclock --show

could you try to redirect the RTC interrupt and IRQ0 to CPU#0? You can 
configure it via /proc/irq/8/smp_affinity and /proc/irq/0/smp_affinity. 
They default to 00000003. (it's an affinity bitmask - the bits for CPU#0 
and CPU#1 are set.) To bind it to CPU#0, set it to 00000001.

this way we should have all concievable RTC related activities on CPU0, 
and we should have the same/similar access patterns as you got with 
maxcpus=1. In theory.

in your current log the interrupts are well-spread out:

           CPU0       CPU1
  0:         75      23663   IO-APIC-edge      timer

so maybe the lockup is related to getting an RTC interrupt (or IRQ#0) on 
CPU#1 while the CMOS access is ongoing on CPU#0 - or something like 
this.

> as long as you made at least one successful hwclock command
I didn't. Simply because I wasn't able to make even _one_ successful call to hwclock --show  since I set maxcpus=2 again.

Restricting the irqs does not seem to work properly:
With irq8 there is no problem, but when I do:
echo 1 > /proc/irq/0/smp_affinity
I get: write error: Input/output error

Restricting irq8 alone did not work.

Just when I posted it, I made the first successful hwclock-call. But nothing else to be reported.

> With irq8 there is no problem, but when I do:
> echo 1 > /proc/irq/0/smp_affinity
> I get: write error: Input/output error
> 
> Restricting irq8 alone did not work.

ok. i forgot that we disallow the restricting of IRQ0.

if you temporarily disable the "irqbalance" service via ntsysv and 
reboot, do all IRQs end up on CPU#0? (you have CONFIG_IRQBALANCE 
disabled, so they should end up there)

so once all IRQs are on CPU#0, does the lockup go away?

> ok. i forgot that we disallow the restricting of IRQ0.
> 
> if you temporarily disable the "irqbalance" service via ntsysv and 
> reboot, do all IRQs end up on CPU#0? (you have CONFIG_IRQBALANCE 
> disabled, so they should end up there)

hm, this might still allow the hw to balance interrupts.

you couldnt set IRQ#0's mask due to the "IRQF_NOBALANCING" flag in 
arch/x86/kernel/time_64.c's 'struct irqaction irq0' initializer. (around 
line 100) Could you remove it from there, and can you set 
/proc/irq/0/smp_affinity with that change in place?

or if you run the 32-bit kernel, the IRQF_NOBALANCING flag is in 
arch/x86/mach-default/setup.c, around line 186.

echo 1 > /proc/irq/0/smp_affinity
echo 1 > /proc/irq/0/smp_affinity
hwclock --show
... freeze ...

Even when I add taskset -01 $$ in front of those commands it freezes.

> echo 1 > /proc/irq/0/smp_affinity
> echo 1 > /proc/irq/0/smp_affinity
> hwclock --show
> ... freeze ...

(i suspect one of the lines was for IRQ#8, right?)

> Even when I add taskset -01 $$ in front of those commands it freezes.

that's weird. To make it even more sure that absolutely nothing runs on 
CPU#1, could you try to boot via init=/bin/bash, that will make bash the 
init task and nothing else runs. (and no consoles, no nothing) Then 
please bind everything to CPU#0 and try to get it to lock up. (Warning: 
if you boot like this then Ctrl-C might not work - so only run commands 
that you know will finish within a finite amount of time.)

also, for good measure, could you bind all interrupts to CPU#0, via:

 for N in /proc/irq/*/smp_affinity; do echo 01 > $N; done

with all these measures in place i cannot think of anything else that 
would run on the second core and that would/could somehow affect the 
southbridge/CMOS.

It seems that taskset + setting all interrupts to smp_affinity=1 did the job already. I will double check this now without taskset and again with only irq0 and irq8 set to smp_affinity=1.

I don't need taskset to eliminate the freeze, but it seems that I need to restrict an irq above 10 to succeed. I try now to determine all irqs that matter.

Determining the irqs seems to be more difficult than I thought. When I make irq23 (nv_sata) able to use both cpus this does not matter until there is disk i/o.
I could run hwclock --show 100 times, but when I did a "du -sh /" in another terminal it froze within a second.

The disk i/o does not matter when irq23 is restricted.

Hm, for now this is all I can get reliably:
It never failed with taskset and all irqs restricted.
I did modify my init scripts to run taskset and your for loop before hwclock is called. It froze several times since then, but this may be due to me lacking bash knowledge.
When I activate irqs it sometimes freezes and sometimes not. When it freezes then at the first call to hwclock --show after a modification to the restrictions. 

Except the observation above there is nothing reliable.

I was just reminded of my first attempt to solve this bug: After some research on the internet I suspected acpi to be responsible for the freezes. I just tried to switch acpi=off with 2.6.24_rc3 and indeed the freeze did not occur.

So there are now 2 possibilities to eliminate the freeze:
1. taskset -p 01 $$ + for N in /proc/irq/*/smp_affinity; do echo 01 > $N; done
2. setting acpi=off as a kernel parameter

> I was just reminded of my first attempt to solve this bug: After some 
> research on the internet I suspected acpi to be responsible for the 
> freezes. I just tried to switch acpi=off with 2.6.24_rc3 and indeed 
> the freeze did not occur.
> 
> So there are now 2 possibilities to eliminate the freeze:
> 1. taskset -p 01 $$ + for N in /proc/irq/*/smp_affinity; do echo 01 > $N;
> done
> 2. setting acpi=off as a kernel parameter

hm. Do you still have two cores after acpi=off? And are all irqs going 
to both CPUs in that case? If not then acpi=off might just be a superset 
of the affinity setting.

I just forgot to mention:

3. setting maxcpus=1

With acpi=off there are still two processors, but both seem to get the id 0. (/proc/cpuinfo says "processor: 0" two times). So no luck there.

Is there anything else that I can test?

I now made a script that:
1. randomly chooses an interrupt
2. sets all other interrupts to smp_affinty=1
3. sets chosen interrupt to smp_affinity=3
4. runs 20 random hwclock commands
5. starts again at 1. as long as it doesn't freeze 

The table below shows how often an interrupt was tested and with what interrupt the test froze.

IRQ| test series
0 : 0 0 2 0 2
1 : 0 0 1 0 2
2 : 0 0 2 0 0
3 : 1 0 2 0 2
4 : 2 0 1 0 0
5 : 2 0 1 0 0
6 : 1 0 1 1 1
7 : 2 0 1 0 0
8 : 0 0 5 0 1
9 : 1 0 3 0 1
10: 1 0 2 1 0
11: 4 0 0 0 1
12: 2 0 2 0 0
13: 3 1 0 0 1
14: 2 0 2 0 3
15: 2 0 1 0 2
18: 1 0 3 0 0
19: 0 f f f 1
20: 0 0 1 0 0
21: 0 0 1 0 1
23: f 0 1 0 f

The digits are the numbers of times it was tested "good". f means it failed during the test for this IRQ.

Since 19 is eth0 and it failed so often, I made the fifth test with "ifconfig eth0 down" and it seems it helped.

My current hypothesis is that it does not depend on the special IRQ, but it freezes as soon as there is any interrupt handled by CPU#1.
In my case this would naturally be the ethernet card and the disk. After shutting eth0 down the next run with it unbound was successful (fifth column).

> My current hypothesis is that it does not depend on the special IRQ, 
> but it freezes as soon as there is any interrupt handled by CPU#1. In 
> my case this would naturally be the ethernet card and the disk. After 
> shutting eth0 down the next run with it unbound was successful (fifth 
> column).

hm. So a possibly safe method of reading the CMOS on your box would be 
to first turn off irqs on all CPUs, then read the CMOS register, then 
re-enable interrupts again? arch/x86/kernel/cpu/mtrr/main.c has such 
code (the ipi_handler() stuff).

> With acpi=off there are still two processors, but both seem to get the 
> id 0. (/proc/cpuinfo says "processor: 0" two times). So no luck there.

ouch! Could you post the full dmesg form such an acpi=off bootup? 
(please also add apic=verbose so that we get all the details)

I just saw that the cpuinfo is the same when I do a normal start. The 2 CPUs differ in the core_id field so everything seems alright. I've just never seen it since even a core2 processor is recognized with two different processor ids. It seems to be different for an athlon X2.

I can upload the dmesg-output though, if you are still interested.

For the other thing. I don't quite get what you mean. My insights into kernel hacking end somewhere near applying patches ...

What chances are there that this is a hardware defect, instead of a kernel problem?

> What chances are there that this is a hardware defect, instead of a 
> kernel problem?

my guess would be hardware defect or BIOS upgrade needed. We might be 
able to work it around by taking the ioapic_lock when doing CMOS ops - 
but this definitely does not have the appearance of a real kernel bug - 
it triggers very easily on your box and does not seem to trigger on 
other boxes.

> my guess would be hardware defect or BIOS upgrade needed.
I'll try to update the BIOS. The only thing is: This doesn't seem to be a problem with WinXP so it would be hard to get it repaired.

Created attachment 13936 [details]
RTC / IO-APIC workaround

Could you try this attached patch, against 2.6.24-rc4-ish kernels?

It's a "shot in the dark" workaround attempt, serializing IO-APIC accesses and RTC accesses.

Does this make any difference to the lockup?

Created attachment 13937 [details]
RTC / IO-APIC workaround, try #2

updated patch - should build on 64-bit as well and should build with RTC as a module to.

Thanks for the patch! I will try it as soon as possible.

Since I still have a warranty for this laptop I will try my luck with HP. It really seems to be a hardware defect:
- There is probably more than one laptop around that has this motherboard/cpu combination and is running linux just fine.
- Also such hardware limitations do not seem sane for a dual-core system.

Thanks very much for taking time to guide me through testing this out. ;)

> Thanks for the patch! I will try it as soon as possible.

it's really just a partial attempt: the workaround only excludes IO-APIC 
sequences - there are other side-effects of executing interrupts on 
another core that this does not impact. The only truly reliable way of 
getting rid of all side-effects is to bind all irqs to the first core - 
but that is not a reasonable limitation for a general purpose OS (of 
course). In the patch i also reduced the polling frequency of the 
is_rtc_updating() call. Which btw. seems a bit buggy to me:

        while (rtc_is_updating() != 0 && jiffies - uip_watchdog < 2*HZ/100)
                mdelay(1);
        ...
        spin_lock_irqsave(&rtc_lock, flags);

i'm not quite sure what rtc_is_updating() is supposed to do - it does 
this:

        uip = (CMOS_READ(RTC_FREQ_SELECT) & RTC_UIP);

if this is a "you must not read the CMOS when the RTC_UIP bit is set" 
restrictition then the current Linux code that uses this bit is buggy. 
There's a (relatively low) chance that your lockup is related to that. 
I'll do a second patch too that also solves this issue. But we've had 
this slightly racy way of testing for RTC_UIP for ages, so i doubt it 
can cause _that_ many problems. we'll see.

> Created an attachment (id=13937) [details]
> RTC / IO-APIC workaround, try #2

This does not fix it.

>         uip = (CMOS_READ(RTC_FREQ_SELECT) & RTC_UIP);
> 
> if this is a "you must not read the CMOS when the RTC_UIP bit is set" 
> restrictition then the current Linux code that uses this bit is buggy. 
> There's a (relatively low) chance that your lockup is related to that. 
> I'll do a second patch too that also solves this issue. But we've had 
> this slightly racy way of testing for RTC_UIP for ages, so i doubt it 
> can cause _that_ many problems. we'll see.

i think the chance for that to be related to the lockup is zero: the 
CMOS updates itself once per second (it is a persistent, battery-backed, 
seconds granularity clock), and the UIP (Update In Progress) bit is set 
when such an update occurs. But an interrupt on another core has no CMOS 
side-effects at all.

Created attachment 13938 [details]
RTC test patch

Please try this patch (rtc-hack-2.patch) too.

this turns off all CMOS accesses within the rtc_get_rtc_time() function. This breaks hwclock in that an incorrect date will be returned - but it should allow us to test the theory that these CMOS accesses are the ones that are causing the lockup.

lets try this to make sure we are on the right track.

> > RTC / IO-APIC workaround, try #2
> 
> This does not fix it.

ok. Lets forget about this patch then and try the second patch, just to 
reassure ourselves that it's indeed those CMOS_READ sequences that cause 
the lockup.

i noticed this in your bootlog:

  AMD Turion(tm) 64 X2 Mobile Technology TL-52 stepping 02
  Brought up 2 CPUs
  testing NMI watchdog ... OK.

While i asked you to enable it to get a debug dump from the wedged 
system, i'd now suggest to turn off the NMI watchdog, to make sure it 
does not interact. The NMI code makes CMOS accesses in certain 
circumstances.

In the meanwhile I tried whether updating to the newest BIOS version helps, but it did not.

The freeze still occurs with both patches applied (workaround+test). (A make clean should suffice to guarantee that the patch is applied in the binary too, doesn't it).

I have to make a break with testing now. But I will test this again soon to be sure that I didn't make any mistakes.

> The freeze still occurs with both patches applied (workaround+test). 
> (A make clean should suffice to guarantee that the patch is applied in 
> the binary too, doesn't it).

yeah, make clean is more than enough. (usually make will pick up any 
changed source files and rebuild any dependencies)

does hwclock --show return some weird date? That should make sure that 
the test patch is applied too.

to make sure that rtc_get_rtc_time() has no effect, could you edit 
drivers/char/rtc.c's rtc_get_rtc_time() function and add this line to 
the 6th line of it:

	return;

(i.e. return early from this function without doing anything)

> (i.e. return early from this function without doing anything)

if with this hack it's still locking up then please repeat the strace 
experiment to figure out in which ioctl it is locking up.

So now it gets strange:
I checked 3 times that the patches are applied, but the clock still gave the right time. Then I checked the kernel configuration and saw that HPET was activated. First i tried to deactivate RTC (and only have HPET) with the result that hwclock didn't find /dev/rtc anymore. Then I deactivated HPET completely and activated RTC again (now built-in not as module) and now the patch seems to do its work.

With the hack applied I'm unable to boot properly because hwclock waits until a second passes, but with your patch it never does. So it's stuck in an endless loop.

I unapplied the second patch then and tried it only with the first patch applied this way and it still froze.

I just did another test: To reassure myself that this is not caused by a broken toolchain, I recompiled the kernel with a LiveCD and the result is the same. So it's unlikely that this is caused by the toolchain.
What else would I have to test to cancel out everything except the kernel and the hardware as a cause?

See bug 9511. This should be the same problem.

> See bug 9511. This should be the same problem.

oh my gosh! Indeed:

   ------- Comment #10 From David P. Reed 2007-12-13 18:49:04 -------
   On my AMD64x2 machine, I discovered the cause is the "outb" to port 
   0x80 done in the inb_p calls used in the CMOS_READ and CMOS_WRITE 
   macros.

could you just try the following, remove this line from 
include/asm-x86/io_32.h and include/asm-x86/io_64.h:

        asm volatile("outb %%al,$0x80" : : : "memory");

does that give you a non-freezing hwclock?

Created attachment 14032 [details]
x86 IO delay fix

Could you try this patch? It replaces port 0x80 access with an udelay(1).

Created attachment 14033 [details]
x86 IO delay fix, take #2

Updated patch attached.

Created attachment 14036 [details]
x86 IO delay fix, take #3

Updated patch - this one is against the upstream 2.6.24-rc-ish kernel. (previous ones were against x86.git)

I sent the laptop back for repairs as I am pretty sure that it the freezes are at least partially hardware related. Two days ago it froze a few times during different tasks (ie. once when running windows ...).
So it will take 4-6 weeks until I can try the new kernel patch.

We suspect that the hwclock lockups are related to the port 80 access related problem debugged by David P. Reed & co on lkml. (see the "[PATCH] x86_64: fix problems due to use of "outb" to port 80 on some AMD64x2 laptops, etc." thread and other related threads)

Please attach the output of dmidecode so that we can add the boot-time workaround for this problem.

Created attachment 14095 [details]
x86: IO delay quirk

Please apply the attached patch to 2.6.24-rc5 and boot with the new io_delay=0xed boot option - does it resolve the problem? (using the DMI info we can make this boot selection automatic)

*** Bug 9511 has been marked as a duplicate of this bug. ***

Created attachment 14553 [details]
dmidecode output

Good morning!

I tested the patch with 2.6.24-rc5 and it seems to work. I enabled the io_delay=0xed option in .config, so the bug is gone with and without the boot-parameter.
I attached the dmidecode output for my laptop.

Does this patch also apply to newer rc-releases?

Keep up the great work!

It looks like the patch has been committed, so the bug can be closed.
Thanks.

yes, we pushed this category of fixes upstream, via:

  commit b02aae9cf52956dfe1bec73f77f81a3d05d3902b
  Author: Rene Herman <rene.herman@gmail.com>
  Date:   Wed Jan 30 13:30:05 2008 +0100

      x86: provide a DMI based port 0x80 I/O delay override.

and subsequent patches. We can still add more DMI quirks if dmidecode is 
reported to this bugzilla. I just added the one reported by 
mereandor@gmail.com.