[olug] 2.4 kernel/ SMP and APIC

Wed Dec 11 23:22:11 UTC 2002

see response below..

On Wed, 11 Dec 2002 15:43:31 -0600
"VincentR" <vincentr at cox.net> wrote:

|Booting with the noapic option would certainly stop the error messages.  
|If it's not a really busy system, you shouldn't see degraded performance.

|Are there any errors listed in /proc/interrupts ?
|
|[root at --- root]# grep ERR /proc/interrupts
|ERR:          0

liberty:/var/log # grep ERR /proc/interrupts
ERR:      14882

Hrmm... I dunno if that's cummulative or what?  I only use this as a
destkop workstation, but my uptime is:
liberty:/var/log # uptime
  5:16pm  up 112 days,  6:36,  1 user,  load average: 0.09, 0.16, 0.25

Is that many interrupts a bad thing???  My CPUs taking a beating?

it's actually in my /var/log/warn logfile... this is the only computer I
know of that has such a log, but it doesn't bother me -- I never look at
it. ;) and those APIC error's are literally the only traffic in there...
albeit it is quite frequent, about 30 - 130 every day.
=(

|:)
|
|----- Original Message ----- 
|From: "Brian Wiese" <bwiese at cotse.com>
|To: <olug at olug.org>
|Sent: Wednesday, December 11, 2002 12:31 PM
|Subject: Re: [olug] 2.4 kernel/ SMP and APIC
|
|
|> is there anyway to stop this kind of stuff from filling up my
|> /var/log/warn ?
|> 
|> ...
|> Oct 21 11:06:37 liberty kernel: APIC error on CPU0: 02(02)
|> Oct 21 11:27:25 liberty kernel: APIC error on CPU0: 02(02)
|> Oct 21 11:32:16 liberty kernel: APIC error on CPU1: 04(04)
|> Oct 21 11:32:16 liberty kernel: APIC error on CPU0: 02(08)
|> Oct 21 11:38:08 liberty kernel: APIC error on CPU0: 08(08)
|> Oct 21 11:38:08 liberty kernel: APIC error on CPU1: 04(04)
|> 
|> all the time. running:
|> Linux liberty 2.4.10-64GB-SMP #1 SMP Fri Sep 28 17:26:36 GMT 2001 i686
|> unknown
|> on SuSE 7.3 pro, dual celeron 366s on abit bp6 ?
|> 
|> I searched online once before, and I don't think there was any
solution,
|> just that it's a common hardware problem that happens?  Doesn't seem to
|> slow me down any... but I wouldn't know either way, it's just always
done
|> this. =(
|> 
|> peace
|> Brian
|> 
|> On Tue, 10 Dec 2002 17:11:03 -0600
|> "VincentR" <vincentr at cox.net> wrote:
|> 
|> |This is mostly an FYI thing for thought...
|> |There's a little used feature in the 2.4 kernel which enables you to
bind
|> an
|> |IRQ to a specific CPU.  Why would you do this?  In my case it was
because
|> |UDP packets in a stream were getting out of sequence due to the APIC
|> |allowing multiple CPU's to service interrupts on the same ethernet
|> device.
|> |You could just turn off APIC with boot loader options
(append="noapic"),
|> but
|> |that would disable most of the advantages of an SMP system.  You could
|> also
|> |increase the number of packets serviced within an interrupt if your
|> driver
|> |allows that configuration in modules.conf (options 3c59x
|> |max_interrupt_work=60).  That helps, but it doesn't fix the problem.
|> |Here's an init script to set the smp_affinity for each network device
|> (word
|> |wrap will probably screw it up):
|> |
|> |[root at --- /root]# cat /etc/init.d/eth_affinity
|> |#!/bin/bash
|> |#
|> |# eth_affinity
|> |#
|> |# chkconfig: 345 15 85
|> |# description: Set IRQ affinity for eth devices.
|> |
|> |# Define which networks get which CPU.
|> |# You must use the first three octets.
|> |CPU0="10.100.116 10.5.7 10.5.9 10.129.7 10.0.9"
|> |CPU1="10.10.10 10.10.11 10.10.0"
|> |
|> |# Where is /proc?
|> |PROC=/proc
|> |IPV4=/proc/sys/net/ipv4
|> |
|> |# Got root?
|> |if [ `id -u` -gt "0" ]; then
|> |        echo "You are not root!"
|> |        exit 1
|> |fi
|> |# Check kernel version.
|> |if [ ! `uname --release | cut -d. -f1-2` = "2.4" ]; then
|> |        echo "You are not running a 2.4 kernel"
|> |        echo "Version detected: "`uname --release`
|> |        exit 1
|> |fi
|> |# Is this an SMP kernel?
|> |if [ ! -f /proc/1/cpu ]; then
|> |        echo "Get an SMP system to do this on!"
|> |        exit 1
|> |fi
|> |# Check for the required files and directories.
|> |if [ ! -d $PROC/irq ] || [ ! -d $IPV4/conf ]; then
|> |        echo "I couldn't find a required file or directory;
Exiting..."
|> |        file $PROC/irq $IPV4/conf
|> |        exit 1
|> |fi
|> |
|> |#DEBUG SECTION##################################################
|> |# Set DEBUG=1 to test this in /tmp.
|> |DEBUG=0
|> |if [ "$DEBUG" = "1" ]; then
|> |        PROC=/tmp
|> |        IPV4=/tmp/ipv4
|> |fi
|> |# Copy files to /tmp if needed.
|> |if [ ! -d $PROC/irq ] || [ ! -d $IPV4/conf ]; then
|> |        cp -a /proc/irq /tmp
|> |        cp -a /proc/sys/net/ipv4 /tmp > /dev/null 2>&1
|> |fi
|> |#DEBUG SECTION##################################################
|> |
|> |# Here we go...
|> |case "$1" in
|> |  start)
|> |        # Find each eth's irq and network; set the affinity.
|> |        for ETH in `/bin/ls $IPV4/conf | egrep -v all\|default\|lo`;
do
|> |                IRQ=`ifconfig $ETH | grep Interrupt | awk -F: '{print
|> $2}' |
|> |awk '{print $1}'`
|> |                NET=`ifconfig $ETH | grep "inet addr" | awk -F:
'{print
|> $2}'
|> || cut -d. -f1-3`
|> |                for net in `echo $CPU0`; do
|> |                        if [ "$NET" = "$net" ]; then
|> |                                echo 1 > $PROC/irq/$IRQ/smp_affinity
|> |                        fi
|> |                done
|> |                for net in `echo $CPU1`; do
|> |                        if [ "$NET" = "$net" ]; then
|> |                                echo 2 > $PROC/irq/$IRQ/smp_affinity
|> |                        fi
|> |                done
|> |        done
|> |        touch /var/lock/subsys/eth_affinity
|> |        $0 status
|> |        ;;
|> |  stop)
|> |        # Find each eth's irq; unset the affinity.
|> |        for ETH in `/bin/ls $IPV4/conf | egrep -v all\|default\|lo`;
do
|> |                IRQ=`ifconfig $ETH | grep Interrupt | awk -F: '{print
|> $2}' |
|> |awk '{print $1}'`
|> |                echo ffffffff > $PROC/irq/$IRQ/smp_affinity
|> |        done
|> |        rm -f /var/lock/subsys/eth_affinity
|> |        $0 status
|> |        ;;
|> |  status)
|> |        # Find each eth's irq and network; display the affinity.
|> |        for ETH in `/bin/ls $IPV4/conf | egrep -v all\|default\|lo`;
do
|> |                IRQ=`ifconfig $ETH | grep Interrupt | awk -F: '{print
|> $2}' |
|> |awk '{print $1}'`
|> |                NET=`ifconfig $ETH | grep "inet addr" | awk -F:
'{print
|> $2}'
|> || cut -d. -f1-3`
|> |                AFFINITY=`cat $PROC/irq/$IRQ/smp_affinity`
|> |                if [ "$AFFINITY" = "00000001" ]; then
|> |                        echo "$ETH IRQ $IRQ has affinity with CPU0 on
|> |network $NET."
|> |                fi
|> |                if [ "$AFFINITY" = "00000002" ]; then
|> |                        echo "$ETH IRQ $IRQ has affinity with CPU1 on
|> |network $NET."
|> |                fi
|> |                if [ "$AFFINITY" = "ffffffff" ]; then
|> |                        echo "$ETH IRQ $IRQ has affinity with ALL on
|> network
|> |$NET."
|> |                fi
|> |                if [ ! "$AFFINITY" = "00000001" ] && [ ! "$AFFINITY" =
|> |"00000002" ] && [ ! "$AFFINITY" = "ffffffff" ]; then
|> |                        echo -e "$ETH IRQ $IRQ network $NET has an
|> affinity
|> |of $AFFINITY."
|> |                        echo "I have no idea what this means!"
|> |                fi
|> |        done
|> |        ;;
|> |  *)
|> |        echo "Usage: eth_affinity {start|stop|status}"
|> |        exit 1
|> |esac
|> |#EOF
|> |
|> |
|> |_______________________________________________
|> |OLUG mailing list
|> |OLUG at olug.org
|> |http://lists.olug.org/mailman/listinfo/olug
|> |
|> 
|> 
|>   Brian Wiese | bwiese at cotse.com | aim: unolinuxguru
|> ------------------------------------------------------
|>   GnuPG/PGP key 0x1E820A73 | "FREEDOM!" - Braveheart 
|> ------------------------------------------------------  
|> This is not about Napster or DVDs. It's about your Freedom.
|>   I'll see your DMCA and raise you a First Amendment.
|>               http://www.anti-dmca.org
|> _______________________________________________
|> OLUG mailing list
|> OLUG at olug.org
|> http://lists.olug.org/mailman/listinfo/olug
|
|
|_______________________________________________
|OLUG mailing list
|OLUG at olug.org
|http://lists.olug.org/mailman/listinfo/olug
|

  Brian Wiese | bwiese at cotse.com | aim: unolinuxguru
------------------------------------------------------
  GnuPG/PGP key 0x1E820A73 | "FREEDOM!" - Braveheart 
------------------------------------------------------  
This is not about Napster or DVDs. It's about your Freedom.
  I'll see your DMCA and raise you a First Amendment.
              http://www.anti-dmca.org