We've got a problem with how the IPL is set in hardirqs. All handlers are run with IPL set to 31, i.e. with all the interrupts masked, which makes interrupt latencies unacceptable. This rework addresses the problem by carefully managing the IPL in the handlers. This is loosely based on an approach taken for one of the Alpha platforms where a silicon erratum prevents the usual mask-and-ack approach from working. We are in a similar position as for most of our platforms we do not have an interrupt mask register and all the interrupt scheduling is based on the IPL. The basic idea is to make interrupt masking to set the IPL to the max and interrupt unmasking to the original IPL of the handler. The latter is saved ASAP upon entry and further interrupts are masked like for other platforms. Later on interrupts may be unmasked based on the absence of the IRQF_DISABLED flag at the time the corresponding handler had been installed. To prevent looping because of the original interrupt request still being asserted the IPL must not be lowered below the original value. Once the handler has run interrupts are masked again. At this point the interrupt source has been acked by the driver so it is safe to lower the IPL so as to permit all the hardware interrupts. Therefore the saved IPL is set appropriately -- I have chosen the highest software interrupt setting so that actual software interrupts, if we want to use them at one point in the future, work as expected. If any softirqs are run at this stage, they will unmask hardware interrupts the usual way and per IPL setting let all of them come. Finally, at the conclusion of the interrupt handler the saved IPL is replaced with one to become in effect after the REI instruction has restored the previous state. The exception handler has been adjusted accordingly. Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org> --- Comments are welcome. I'll think yet about unhandled_exception() and machine_check_handler() -- these probably just need to run at max IPL throughout. Jan-Benedict, what's your status on the tree update? If the patch needs an update, I'll be happy to make it fit into your current tree; otherwise please apply. We might think about switching to generic hardirq handlers at some point too; I will need to get a better understanding of how our IRQ probes work before I feel ready to work on it though. Maciej patch-mips-2.6.18-20060920-vax-irq-3 Index: linux-mips-2.6.18-20060920-cougar/arch/vax/kernel/entry.S =================================================================== --- linux-mips-2.6.18-20060920-cougar.orig/arch/vax/kernel/entry.S +++ linux-mips-2.6.18-20060920-cougar/arch/vax/kernel/entry.S @@ -4,6 +4,7 @@ * Copyright 2000, Kenn Humborg <kenn@linux.ie> * 2001 atp. Additions for Machine check handling. * Copyright 2004, 2005 Jan-Benedict Glaw <jbglaw@lug-owl.de> + * Copyright (c) 2009 Maciej W. Rozycki */ #include <linux/sys.h> @@ -50,6 +51,10 @@ ENTRY(irqvec_handler) * irqvector structure (at our "return address") */ + /* Save current IPL for interrupt unmasking and mask interrupts. */ + mfpr $PR_IPL, __min_ipl + mtpr $31, $PR_IPL + pushl %r0 /* Push R0 to have a free working register */ /* @@ -196,10 +201,13 @@ ret_from_syscall: * +8 saved PSL */ - /* Restore R0 and dismiss exception */ - movl (%sp)+, %r0 + movl (%sp)+, %r0 /* Restore R0. */ + + /* Retrieve previous IPL from saved PSL for interrupt unmasking. */ mtpr $31, $PR_IPL - rei + extzv $16, $5, 4(%sp), __min_ipl + + rei /* That's it! */ ENTRY(ret_from_fork) Index: linux-mips-2.6.18-20060920-cougar/arch/vax/kernel/interrupt.c =================================================================== --- linux-mips-2.6.18-20060920-cougar.orig/arch/vax/kernel/interrupt.c +++ linux-mips-2.6.18-20060920-cougar/arch/vax/kernel/interrupt.c @@ -5,9 +5,11 @@ * need to be a separate interrupt stack per-cpu, within the * per-cpu data structures. * + * Copyright (c) 2009 Maciej W. Rozycki + * * FIXME: We should use the new interrupt architecture. It looks like * a closer match to the VAX SCB. -*/ + */ #include <linux/types.h> #include <linux/errno.h> @@ -53,6 +55,13 @@ static struct stray_handler stray_handle /* Non-zero when autoprobing interrupt vectors */ static int autoprobing; +/* + * Minimum IPL during interrupt processing. Values other than the + * current IPL may be written only while interrupts are disabled + * (IPL set to MAX). For SMP this will have to be defined per CPU. + */ +int __min_ipl; + void guard_int_stack(void) { @@ -414,41 +423,40 @@ static inline void dispatch_irq(struct p vec_num = vec->vec_num; kstat_cpu(smp_processor_id()).irqs[vec_num]++; + + if (!(action->flags & IRQF_DISABLED)) + local_irq_enable(); + action->handler(vec_num, action->dev_id, regs); - if (action->flags & SA_SAMPLE_RANDOM) + if (action->flags & IRQF_SAMPLE_RANDOM) add_interrupt_randomness(vec_num); + + local_irq_disable(); } /* - * This is called once we know that an interrupt or exception is actually - * an interrupt. + * Call the appropriate hardirq handler. Once all the hardirq processing + * has been done adjust the IPL saved for interrupt unmasking suitably for + * softirq handling. We could actually send ourselves a software interrupt + * instead, which would be taken straight away once we have got back to the + * process level, but that would only incur unnecessary overhead with no + * gain in return. */ static inline void do_irq(struct pt_regs *regs, struct irqvector *vec) { - int flags; - - /* Fake a single-priority-level interrupt system by raising IPL - to 31 for _any_ interrupt. This is such a waste of the VAX's - hardware capabilities... */ - irq_enter(); - - local_irq_save(flags); - dispatch_irq(regs, vec); + __min_ipl = IPL_SWF; irq_exit(); - - local_irq_restore(flags); - - if (local_softirq_pending()) - do_softirq(); } static inline void do_exception(struct pt_regs *regs, struct irqvector *vec, void *excep_info) { kstat_cpu(smp_processor_id()).irqs[vec->vec_num]++; + local_irq_enable(); vec->excep_handler(regs, excep_info); + local_irq_disable(); } /* Index: linux-mips-2.6.18-20060920-cougar/arch/vax/kernel/time.c =================================================================== --- linux-mips-2.6.18-20060920-cougar.orig/arch/vax/kernel/time.c +++ linux-mips-2.6.18-20060920-cougar/arch/vax/kernel/time.c @@ -2,6 +2,7 @@ * Copyright (C) 1995 Linus Torvalds * VAX port copyright atp 1998. * (C) 2000 Erik Mouw <J.A.K.Mouw@its.tudelft.nl> + * Copyright (c) 2009 Maciej W. Rozycki * * 22-oct-2000: Erik Mouw * Added some simple do_gettimeofday() and do_settimeofday() @@ -59,7 +60,8 @@ void __init time_init(void) wall_to_monotonic.tv_sec = -xtime.tv_sec; wall_to_monotonic.tv_nsec = -xtime.tv_nsec; - if (request_irq(0x30, do_timer_interrupt, 0, "timer", NULL)) { + if (request_irq(0x30, do_timer_interrupt, IRQF_DISABLED, + "timer", NULL)) { printk("Panic: unable to register timer interrupt handler\n"); HALT; } Index: linux-mips-2.6.18-20060920-cougar/include/asm-vax/system.h =================================================================== --- linux-mips-2.6.18-20060920-cougar.orig/include/asm-vax/system.h +++ linux-mips-2.6.18-20060920-cougar/include/asm-vax/system.h @@ -1,3 +1,6 @@ +/* + * Copyright (c) 2009 Maciej W. Rozycki + */ #ifndef _VAX_SYSTEM_H #define _VAX_SYSTEM_H @@ -77,6 +80,42 @@ struct task_struct *__switch_to(struct t #define smp_rmb() rmb() #define smp_wmb() wmb() + +#define IPL_USR 0 +#define IPL_SW1 1 +#define IPL_SW2 2 +#define IPL_SW3 3 +#define IPL_SW4 4 +#define IPL_SW5 5 +#define IPL_SW6 6 +#define IPL_SW7 7 +#define IPL_SW8 8 +#define IPL_SW9 9 +#define IPL_SWA 10 +#define IPL_SWB 11 +#define IPL_SWC 12 +#define IPL_SWD 13 +#define IPL_SWE 14 +#define IPL_SWF 15 +#define IPL_DEV0 16 +#define IPL_DEV1 17 +#define IPL_DEV2 18 +#define IPL_DEV3 19 +#define IPL_DEV4 20 +#define IPL_DEV5 21 +#define IPL_DEV6 22 +#define IPL_DEV7 23 +#define IPL_ERR0 24 +#define IPL_ERR1 25 +#define IPL_ERR2 26 +#define IPL_ERR3 27 +#define IPL_ERR4 28 +#define IPL_ERR5 29 +#define IPL_POWERFAIL 30 +#define IPL_MAX 31 + +extern int __min_ipl; + #define getipl() __mfpr(PR_IPL) #define setipl(ipl) __mtpr(ipl, PR_IPL) @@ -97,32 +136,14 @@ struct task_struct *__switch_to(struct t __r0; \ }) -#define local_save_flags(flags) ((flags) = getipl()) -#define local_irq_save(flags) ((flags) = swpipl(31)) -#define local_irq_restore(flags) setipl(flags) -#define local_irq_disable() setipl(31) - -#define irqs_disabled() (__psl.ipl == 31) - -/* If we're handling an interrupt (i.e. the IS bit is set in the - PSL and we're on the interrupt stack), then we must not enable - interrupts by dropping IPL all the way to 0. If we do, and - another interrupt comes in, then this second interrupt will - be handled normally, but will REI to a PSL with IS set and - an IPL of 0, which REI doesn't like at all. +#define local_irq_disable() setipl(IPL_MAX) +#define local_irq_enable() setipl(__min_ipl) - So, instead, we drop IPL to 1 if we're running on the interrupt - stack, thus making sure that REI will be kept happy. */ +#define local_save_flags(flags) do { (flags) = getipl(); } while (0) +#define local_irq_save(flags) do { (flags) = swpipl(IPL_MAX); } while (0) +#define local_irq_restore(flags) setipl(flags) -extern __inline__ void -local_irq_enable(void) -{ - if (__psl.is) { - setipl(1); - } else { - setipl(0); - } -} +#define irqs_disabled() (getipl() == IPL_MAX) #ifdef __SMP__ # error "SMP not supported" _______________________________________________ Linux-Vax mailing list Linux-Vax@mail.pergamentum.com http://mail.pergamentum.com/mailman/listinfo/linux-vax